DOC  FILE  COPY.  . ADA038294 


ROBUST  IDENTIFICATION  OF  LINEAR  SYSTEMS 


V.  David  VandeLinde 


February  1977 


J Approved  for  public  releast;  distribution  unlimited. 
I 


USA  BALLISTIC  RESEARCH  LABORATORY 

ABERDEEN  PROVING  GROUND,  MARYLAND 


Destroy  this  report  when  it  is  no  longer  needed. 

Do  not  return  it  to  the  originator. 

jt 


Secondary  distribution  of  this  report  by  originating 
or  sponsoring  activity  is  prohibited. 

Additional  copies  of  this  report  may  be  obtained 
from  the  National  Technical  Information  Service, 

U.S.  Department  of  Comnerce,  Springfield,  Virginia 
22151. 


f 

«■ 

* 


4 

The  findings  in  this  report  are  not  to  be  construed  as 
an  official  Department  of  the  Army  position,  unless 
so  designated  b/  other  authorized  documents. 


V 

i 


SECURITY  CLASSIFICATION  OF  THIS  pace  (When  Palm  Enfrrd) 

! REPORT  DOCUMENTATION  PAGE 


nr  REPORT  NUMBER 


|2.  GOVT  ACCESSION  NO 


n brl 


I A.  TITL F (end  Subtitle) 


( ROBUST  IDENTIFICATION  OF  LINEAR  SYSTEMS 

v k - * * » 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 
3 RECIPIENT'S  CATALOG  NUMBER 


5.  type  of  report  & period  covered 


Final 


6.  PERFORMING-  DUG.  BtFC 


/ ' ' y V . David/VandeLindej 

Department  ot  tlectpital  Engineering 

The  Johns  HopkiT^dniversity 

9.  PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

US  Army  Ballistic  Research  Laboratory 
Aberdeen  Proving  Ground,  Maryland  21005 

Is 

It.  CONTROLLING  OFFICE  NAME  AND  ADDRESS 

US  Army  Materiel  Development  6 Readiness  Command-' 
5001  Eisenhower  Avenue 
Alexandria,  Virginia  22335 

1 4.  MONITORING  AGENCY  NAME  & ADDRESS^//  dilfmrmnt  trow  Controlling  Office) 


8.  CONTRACT  OR  GRANT  NUMBERS; 


10.  PROGRAM  ELEMENT.  PROJECT,  TASK 
AREA  6 WORK  UNIT  NUMBERS 


/; 


RDTE  Project  IT] 

)U,  . REPQ&I  .D AX6 -- . 

FEB»«0WLa9“', 

it:  number  of  pages 


15.  SECURITY  CLASS,  (ot  thie  report) 


15a.  DECLASSIFICATION  DOWNGRADING 
SCHEDULE 


I 16.  DISTRIBUTION  STATEMENT  (ot  thie  Report  j 


Approved  for  public  release;  distribution  unlimited. 


I 17.  DISTRIBUTION  STATEMENT  (ot  the  mbetrmct  entered  In  Block  20,  It  different  from  Report) 


16.  SUPPLEMENTARY  NOTF.S 


Work  supported  by  US  Army  Ballistic  Research  Laboratory  through  the  Laboratory 
Research  Cooperative  Program  of  the  Army  Research  Office,  under  Task  Order 
76-20,  Basic  Agreement  DAHC04-72-A-0001 . 


I 19.  KEY  WORDS  (Continue  on  reveree  eide  It  r.eceeemry  end  Identity  by  block  number) 


Robust  Estimation 
System  Identification 
Robust  Statistical  Procedures 
Linear  Systems 


Quantization 

Linear-Quadratic  Estimation  and  Control 
Cross-Correlation  Methods 


S TRACT  (Continue  en.  row 


[ Identity  by  block  number) 


ams/kst 


^^The  first  two  sections  of  this  report  survey  the  current  techniques  of 
identification  and  time  series  analysis  for  linear,  time  invariant,  single- 
input,  single-output  systems  with  noisy  measurements.  Typically,  this  additive 
noise  has  been  modeled  either  as  being  completely  specified  (e.g.,  a Gaussian 
with  known  parameters)  or  as  being  required  to  satisfy  general  conditions  such 
as  ^whiteness’^and  non-correlation  with  the  input.  Departures  from  the 
assumed  noise  model  sometimes  cause  severe  deterioration  in  the  efficiency  of  _>  * 


DD  1473  EDITION  OF  I NOV  SS  IS  OBSOLETE 


UNCLASSIFIED 

SECURITY  CLASSIFICATION  OF  THIS  PAGE  Dmt • Enf»r.rf) 


« >. 


«cumTY  classification  of  this  PAoerwh«B  dm  (him« 


20,  ABSTRACT  (Continued]  *L 

K 

available  identification  algorithms.  Since  it  seems  reasonable  to  have  some, 
if  not  complete,  knbwledge  of  the  operating  environment,  it  is  assumed  in 
this  report  that  the  measurement  noise  w^  has  a distribution 
FCw)  = (l-e)K(w)  + &C(w),  where  K(>)  Is  a completely  specified  distribution 
and  CC*)  belongs  to  some  broad  class  of  distribution.— 

In  the  third  section,  a robust  scheme  for  estimating  the  system  cross 
correlations  is  proposed  in  order  to  desensitize  the  performance  of  the 
identification  algorithm  to  the  distribution  of  w^.  Extensive  computer 
simulations  show  that  the  proposal  provides  a robust  identification  technique 
which  has  good  uniform  behavior  over  a variety  of  distributions  for  wv. 

■’  ■■  rr 


UNCLASSIFIED 

S*CU*lTV  CL  ASS!  Pi  C ATI  ON  OF  THIS  FACEHFh#n  Dmtm  Enfrwd) 


TABLE  OF  CONTENTS 


Page 

. 

SECTION  1.  A SURVEY  OF  IDENTIFICATION  TECHNIQUES  FOR  SINGLE- 
INPUT SINGLE -OUTPUT,  LINEAR,  TIME  INVARIANT,  NOISY 
SYSTEMS  MODELED  IN  DISCRETE  TIME  5 

SECTION  2.  A BRIEF  INTRODUCTION  TO  TIME  SERIES  ANALYSIS  ....  49 

SECTION  3.  ROBUST  ESTIMATION  OF  CROSS  CORRELATIONS  FOR  PARAMETER 


IDENTIFICATION  OF  SINGLE-INPUT,  SINGLE -OUTPUT, 

LINEAR,  TIME  INVARIANT,  NOISY  SYSTEMS  MODELED  IN 
DISCRETE  TIME 83 

APPENDIX  A - ROBUST  QUANTIZATION  OF  DISCRETE-TIME 
SIGNALS  WITH  UNIMODAL  DISTRIBUTIONS  AND  GENERALIZED 
MOMENT  CONSTRAINTS  235 

APPENDIX  B - ROBUST  PROPERTIES  OF  SOLUTIONS  TO 
LINEAR-QUADRATIC  ESTIMATION  AND  CONTROL  PROBLEMS  . . 249 

DISTRIBUTION  LIST 255 


DISTRIBUTION  LIST 255 


W-'W^lgfr— 


SECTION  1 

A SURVEY  OF  IDENTIFICATION  TECHNIQUES  FOR  SINGLE- INPUT 
SINGLE-OUTPUT,  LINEAR,  TIME  INVARIANT,  NOISY  SYSTEMS 
MODELED  IN  DISCRETE  TIME 


Chapter  I General  Introduction  and  Scope  of  the  Report 
Chapter  II  Basic  Elements  of  the  Identification  Problem 

A.  Class  of  Models 

B.  Class  of  Inputs 

C.  Criterion  for  Equivalence 

D.  Type  of  Implementation 

Chapter  III  Off-line  Techniques  for  Identification 

A.  Least  Squares 

B.  Generalized  Least  Squares 

C.  Maximum  Likelihood 

D.  Instrumental  Variables 

Chapter  IV  On-line  Techniques  for  Identification 

A.  On-line  Least  Squares 

B.  On-line  Generalized  Least  Squares 

C.  On-line  Instrumental  Variables 

D.  Stochastic  Approximation 

E.  Correlation-cum-Least  Squares 

Chapter  V Concluding  Remarks 

Bibliography 


r, v„  , >,  , 


£ 


Chapter  I - General  Introduction  and  Scope  of  the  Report 

The  problem  of  identification  arises  as  a natural 
consequence  of  control  engineering,  since  any  control  system 
design  requires  knowledge  of  the  parameters  of  the  system  to 
be  controlled.  The  field  is  quite  vast  at  present,  so  that 
a critical  review  of  state-of-the-art  techniques  must  of 
necessity  be  focussed  on  a specific  problem.  The  scope  of 
this  discussion  will  be  limited  principally  to  popular  iden- 
tification methods  for  linear,  time  invariant,  noisy  systems 
modeled  in  discrete  time.  This  is  not  as  much  of  a restriction 
as  one  might  be  inclined  to  think,  since  the  bulk  of  the 
research  in  identification  has  been  in  this  area.  We  consider 
the  discrete  time  formulation  principally  for  ease  in  handling 
white  noise,  the  modelling  of  which  in  continuous  time  is  a 
technically  difficult  problem.  Another  good  reason  is  that 
such  models  are  naturally  suited  for  digital  computer  simulation 
and  have  found  wide  usage. 

There  are  several  books  and  survey  papers  in  the  field. 
Eykhoff  [1974]  is  the  most  current  and  most  comprehensive  text 
available  at  present;  one  may  also  refer  to  Sage  and  Melsa 
[1971]  and  Graupe  [1972].  For  survey  papers,  Astrom  and 
Eykhoff  [1971]  and  Nieman,  Fisher  and  Seborg  [1971]  may  be 
consulted.  The  latter  has  an  extensive  bibliography.  Pro- 
ceedings of  the  3rd  IFAC  Symposium  on  Identification  and  System 
Parameter  Estimation  [1973]  and  the  special  issue  of  the  IEEE 
Transactions  in  Automatic  Control  [Dec.  1974]  are  quite 


comprehensive  and  up-to-date  in  the  coverage  of  the  different 
aspects  of  identification.  Quite  a few  of  the  papers  serve 
the  dual  purpose  of  providing  concise  surveys  and  highlighting/ 
presenting  new  results.  Eykhoff  [1974],  Astrom  and  Eykhoff 
[1971]  and  the  special  IEEE  issue  [Dec.  1974]  have  been  used 
extensively  in  preparing  this  paper. 

In  the  manner  of  Astrom  and  Eykhoff  [1971]  , the  definition 
of  identification  given  by  Zadeh  [1962]  may  well  be  chosen  as 
our  starting  point: 

"Identification  is  the  determination,  on  the  basis  of 
input  and  output,  of  a system  within  a specified  class  of 
systems,  to  which  the  system  under  test  is  equivalent". 
Identification  is  thus  the  'inverse  problem'  of  system  analysis; 
given  an  input  and  output-time  history,  determine  the  equations 
that  describe  the  behavior  of  the  system.  Using  Zadeh 's 
definition,  an  identification  problem  is  thus  characterized 
by  three  quantities:  a class  of  systems,  S = {s}  , a class 

of  input  signals,  U , and  a criterion  to  determine  equivalence. 
The  system  under  test  will  be  referred  to  as  the  process  and 
the  elements  of  S will  be  called  models . From  the  practical 
viewpoint,  we  believe  it  is  important  to  consider  a fourth 
characteristic  of  any  identification  technique,  viz.  its 
implementation  - on-line  or  off-line.  We  shall  consider 
briefly  the  basic  elements  of  the  identification  problem:  viz., 
the  class  of  models,  the  class  of  input  signals,  the  equivalence 
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criterion  and  the  type  of  implementation.  We  shall  then 
proceed  to  the  task  of  discussing  off-line  and  on-line 
techniques  of  identification  for  linear  time  invariant 
systems  in  stochastic  environments. 
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Chapter  II  - Basic  Elements  of  the  Identification  Problem 
A.  Class  of  Models 

Any  identification  problem  is  basically  formulated  by 
the  choice  of  model  structure.  The  choice  will  depend  not 
only  on  the  a priori  knowledge  available  but  also  on  the 
purpose  of  the  identification.  It  will  greatly  influence 
the  nature  of  the  identification  problem,  such  as:  the  manner 

of  utilizing  the  results  of  the  identification  in  subsequent 
operations,  the  effort  expended  in  computation,  the  possibility 
of  obtaining  unique  solutions  etc.  There  are,  unfortunately, 
few  general  results  available  with  regard  to  choice  of  structure. 

The  principal  choice  seems  to  be  between  parametric  and 
nonparametric  models.  It  might  be  instructive  to  consider  the 
discussion  by  McGhee  [1963],  wherein  he  coins  the  terms  'function 
space  description'  and  'parameter  space  description'.  The 
first  uses  the  idea  of  tranf ormation  defined  over  a function 
space.  The  function  space  provides  a representation  of  the 
process  input  signal.  Examples  of  such  representations  can 
be  found  in  the  Fourier  series  expansion,  the  Laguerre  function 
expansion  etc.  The  transformation  that  is  defined  over  this 
space  follows  from  the  dynamics  of  the  process.  The  process 
output  signal  may  be  represented  on  a similar  space.  In  these 
terms  the  identification  problem  is  to  find  what  transformation 
from  input  function  space  to  output  function  space  characterizes 
the  process.  As  nc  information  about  the  physical  structure 
of  the  process  or  its  assumed  mathematical  equations  is  used, 
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this  approach  is  of  the  'black  box'  identification  type. 

The  'parameter  space  description'  starts  from  an  assumed 
mathematical  description  of  the  process  dynamics.  This 
description  is  a parametric  model  of  finite  dimension.  The 
coordinates  of  the  parameter  space  are  the  numerical  values  ' 
of  the  quantities  that  determine  the  'output'  of  the  model. 

If,  for  example,  the  assumed  description  is  an  ordinary  dif- 
ferential equation,  then  the  coordinates  may  be  the  values  of 
the  coefficients  and  the  initial  conditions.  If  there  is  no 
forcing  funtion  (input) , then  from  this  one  point  in  the 
parameter  space  one  can  predict  the  process  output.  If  there 
is  a forcing  function,  then  the  unknown  parameters  of  that 
signal  increase  the  dimensionality  of  the  parameter  space. 

The  dimensionality  remains  finite,  while  in  principle  an  infinite 
number  of  parameters  has  to  be  determined  in  the  function  space 
description.  Consequently  one  distinguishes  between: 
Nonparametric  models 

e.g.  a)  impulse  responses 

b)  transfer  functions 

c)  spectral  densities 

and 

Parametric  models 

e.g.  a)  differential  (difference)  equations  of  pre- 
determined form  and  order 
b)  state  models 

It  is  known  that  parametric  models  can  give  results  with 
large  errors  if  the  order  of  the  model  does  not  agree  with  the 
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order  of  the  process.  Nonparametric  representations  have  the 
advantage  that  it  is  not  necessary  to  specify  the  order  of 
the  process  explicitly.  These  representations  are,  however, 
intrinsically  infinite  dimensional. 

For  linear  systems,  in  order  to  obtain  unique  solutions 
as  well  as  to  be  able  to  construct  efficient  algorithms  it  is 
of  interest  to  find  representations  of  the  system  which 
contain  the  smallest  number  of  parameters,  viz.  CANONICAL 
representations.  A comprehensive  up-to-date  study  of  canonical 
forms  will  be  found  in  Denham  [1974]  which  contains  also  the 
essential  contributions  of  Popov  [1972] , Weinert  and  Anton 
[1972],  Caines  [1972],  Mayne  [1972  a & b]  and  Kalman  [1973]. 
Also,  Tse  and  Weinert  [1973]  have  reported  a scheme  for  deter- 
mining system  order,  whereas  tests  of  order  for  parametric 
models  have  been  proposed  by  Anderson  [1962],  Astrom  [1968] 
and  Woodside  [1970].  Dickinson,  Kailath  and  Morf  [1974]  expose 
the  interrelations  between  frequency  domain  and  state  space 
descriptions  of  multivariable  linear  systems. 

In  the  case  of  linear  systems,  the  knowledge  of  one 
characteristic  process  time  function  (e.g.  impulse  response) 
is  sufficient  to  determine  process  output  for  arbitrary  input 
signals.  Such  a procedure  has  advantages  which  are  also 
desirable  for  the  description  of  nonlinear  processes  (George 
[1959 ] ) , viz . 

a)  it  gives  an  explicit  input/output  relationship 

b)  it  facilitates  the  discussion  of  combinations  of  systems 

c)  it  allows  the  consideration  of  random  inputs 


12 

9 

V 


. > , 


For  some  classes  of  nonlinear  processes  these  requirements 
are  fulfilled  by  Volterra  series.  The  Volterra  [1959]  series 
method  treats  the  linear  case  as  a sub-case  of  the  nonlinear 
case,  a very  desirable  property  indeed.  Flake  [1963] , Schetzen 
[1974],  Baumgartner  and  Rugh  [1975]  and  Harper  and  Rugh  [1975] 
have  developed  methods  for  identifying  certain  classes  of 
nonlinear  processes  using  Volterra  series. 

As  a concluding  note  on  process  models,  it  may  be  worth- 
while to  mention  the  question  of  identif iability . .This  has 
been  surveyed  by  Glover  and  Willems  [1974]  and  been  discussed 
by  Tse,  Weinert,  Anton  and  Mehra  [1973],  Balakrishnan  [1969] 
and  Staley  and  Yue  [1970]. 

B.  Class  of  Inputs 

It  is  well  known  (and  as  may  be  reasonably  expected)  that 
significant  simplifications  in  computations  can  be  achieved 
by  choosing  input  signals  of  a special  type,  e.g.  impulse 
functions,  step  functions,  'colored'  or  white  noise,  sinusoidal 
signals,  pseudo  random  binary  sequences  (PRBS)  etc.  A bibli- 
ography on  PRBS  is  given  in  Nikiforuk  and  Gupta  [1969] . Refer 
also  to  Godfrey  [1970].  For  the  use  of  deterministic  signals 
see  Strobel  [1968],  VandenBos  [1970],  Welfonder  and  Hasenkcpf 
[1970],  Cumming  [1970].  For  periodic  test  signals,  see 
Vanden3os  [1974]. 

From  the  viewpoint  of  applications  it  would  be  very 
desirable  to  use  techniques  which  do  not  make  strict  limitations 
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on  the  inputs.  However,  if  the  input  signals  can  be  chosen, 
how  should  this  be  done?  The  problem  of  designing  good 
probing  or  test  signals  is  an  important  one  in  many  industrial 
applications  and  Goodwin,  Zarrop  and  Payne  [1974]  discuss 
the  simultaneous  design  of  test  signals,  sampling  intervals 
and  input  filters.  A survey  of  optimal  input  synthesis  is 
done  by  Mehra  [1974]  who  has  also  obtained  new  results  using 
the  statistical  studies  of  Kiefer  and  Wolfwowitz  [1959]  on  ex- 
perimental design  for  regression  problems.  Keviczky  [1975] 
works  along  similar  lines  in  'designing'  optimal  inputs  for 
identification  as  extensions  of  the  work  of  Box  and  Draper 
[1971],  Kiefer  [1961]  and  Wynn  [1970]. 

C.  Criterion  for  Equivalence 

The  criterion  for  determining  equivalence  in  the  definition 
of  the  identification  problem  is  often  a minimization  of  a 
scalar  loss  function.  Mostly,  the  criterion  is  expressed  as 
a functional  of  an  error,  e.g. 

T 

v(y,yM)  = / e2 (t) dt  (1) 

0 

where  y is  the  process  output,  yM  the  model  output  and 
e the  error;  y , yM  and  e are  considered  functions 
defined  on  (0,T).  In  the  case 

e = y - ym  = y - m(u)  (2) 

where  M(u)  denotes  the  output  of  the  model  when  the  input 
is  u , e is  called  the  output  error  (Fig.  1) . 
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It  is  the  natural  definition  when  the  only  disturbances 
are  white  noise  errors  in  the  output  measurements. 

In  the  case 

e = u ~ um  = u ~ M 1 (y)  (3) 

where  u>(1  = M ^ (y)  denotes  the  input  of  the  model  which 
produces  the  output  y , e is  called  the  input  error  (Fig.  2) . 
The  notation  M ^ implies  the  assumption  that  the  model  is 
invertible.  Rigorous  definitions  of  the  concept  of  inverti- 
bility  are  discussed  by  Brockett  and  Mesarovic  [1965]  , 

Silverman  [1969]  and  Sain  and  Massey  [1969] . The  criterion 
(1)  with  error  defined  by  (3)  is  natural  when  the  disturbances 
are  white  noise  entering  at  the  system  input. 

In  the  more  general  case  (Fig.  3)  the  error  can  be 
defined  as 

e = M"X(y)  - Ml(u)  (4) 

where  represents  an  invertible  model. 

This  type  of  model  and  error  are  referred  to  as  generalized 
model  and  generalized  error  (Eykhoff  [1963]). 

A special  case  of  the  generalized  error  is  the  "equation 
error"  introduced  by  Potts,  Ornstein  and  Clymer  [1961] , and 
used  extensively  by  Mendel  [1973]  . 

Another  type  of  identification  criterion  is  obtained  by 
imbedding  the  problem  in  a probabilistic  framework.  If  S 
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is  defined  as  a parametric  class,  S = {Sg}  , where  B is 
a parameter,  the  identification  problem  becomes  a parameter 
estimation  problem,  enabling  the  use  of  the  tools  of  estimation 
and  decision  theory.  However,  in  many  probabilistic  situations 
it  turns  out  that  the  estimation  problem  can  be  reduced  to  an 
optimization  problem,  with  the  loss  function  given  by  the 
probabilistic  assumptions. 

D.  Type  of  Implementation 
(Eykhoff  [1974]) 

All  solutions  to  parametric  identification  problems 
consist  of  finding  the  extremum  of  the  loss  functions  consid- 
ered as  a function  of  the  parameter  6.  A distinction  then 
can  be  made  with  respect  to  the  type  of  implementation. 

Consider  the  correspondence  between  process  and  model  to 

be  established  by  the  error  criterion 

T 

v{y,yM;6}  = / (y (t) -y  (t; 8 ) }2dt 
0 

where  8 = (B1, — , 8n) 


Then  one  can  follow  one  of  two  strategies: 


I. 


Put 


0 for  i 


1 , . . * , n . 


This  is  a necessary  condition  for  obtaining  the  minimal 
error.  These  n equations  with  n unknown  estimates 

can  be  solved  for  the  B's  , thus  providing  explicit 
mathematical  relations  to  obtain  numerical  quantities. 

3 V 

II.  Put  g-r—  0 for  i = l,...,n. 
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A convergence  towards  zero  can  be  obtained  if 


av 

3gi 


can 


be  determined  by  a suitable  instrumentation  and  if  these  values 
are  then  used  for  the  adjustment  of  a physical  model  such 
that  model  characteristics  approach  process  characteristics 
in  some  pre-determined  sense. 

One  can  thus  consider  the  type  I approach  as  basically 
an  OFF-LINE  one-shot  technique  where  the  estimate  (of  3) 

a)  is  available  after  a finite  number  of  elementary  operations 

b)  requires  considerable  memory 

c)  is  not  available  in  an  approximate  form  as  an  intermediate 
result 

d)  is  open  loop  with  respect  to  the  estimate. 


On  the  other  hand,  the  type  II  approach  is  an  iterative 
ON-LINE  procedure  where  the  estimate 

a)  is  available  (in  principle)  after  an  infinite  number  of 
elementary  operations 

b)  requires  less  memory 

c)  is  available  in  an  approximate  form  as  an  intermediate 
result 

d)  is  closed  loop  with  respect  to  the  estimate 

e)  is  found  by  a self  correcting  procedure. 

In  terms  of  engineering  applications,  on-line  schemes 
have  been  favored  heavily  because  of  properties  (b)-(e). 
However,  on-line  methods  usually  suffer  from  the  drawback  of 
just  guessing  initial  estimates. 
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The  rest  of  the  paper  will  be  devoted  to  brief  descriptions 
of  the  popular  identification  techniques  for  discrete  models 
of  linear  time  invariant  systems  in  stochastic  environments. 

The  format  of  presentation  will  be  to  first  describe  off-line 
techniques,  followed  by  a listing  of  on-line  methods.  Quite 
often  it  will  be  seen  that  an  off-line  technique  has  been  put 
into  recursive  form  for  on-line  implementation.  An  effort  has 
been  made  to  state,  for  each  technique,  the  necessary  assump- 
tions and  the  a priori  knowledge  required.  Since  it  is  only 
possible  to  identify  the  controllable  and  observable  parts  of 
a linear  system  from  input/output  data  (Kalman  [1963]),  the. 
is  no  loss  of  generality  to  assume  that  all  processes  of 
interest  in  this  report  are  controllable  and  observable.  In 
addition,  the  processes  are  assumed  to  be  stable.  Further- 
more, we  consider  only  single-input  single-output  systems  for 
convenience  of  representation  and  in  consideration  of  the 
fact  that  it  is  the  most  studied  problem  in  system  identif icatior 


Chapter  III  - Off-line  Techniques  for  Identification 


It  is  logical  to  start  with  the  oldest  technique  - least 
squares  identification  of  a parametric  model. 

A.  Least  Squares 

(Isermann,  Baur,  Bamberger,  Kr.eppo  and  Siebert  ([1974]) 


Process  Model: 

(Fig.  4) 

y (k) +a.y (k-1) +. . . . +a  y (k-n)  = b.  u (k-1) + . . . ,+b  u(k-n)+e(k) 
1 n jl  n 

(6) 

with  y -»■  Process  output  e -*■  Generalized  error 

u - Process  input 


or 


V 

+ e.  where 
k 

ek 

[y  (k) 

y (k-i)  .. 

. .y  (k- 

[ u ( k— 

1)  u (k-2)  .. 

. . u (k 

[ 1 

A 

a,  

A 

. . a ] 

i 

n J 

tr> 

H* 



,.£ni 

and 


(7) 


For  the  actual  process,  because  of  additive  noise  n(k) 
in  the  process  output  y(k),  the  equations  are: 

x (k) +a. x (k-1) +. . . . +a  x (k-n)  = b. u (k-1 )+.... +b  u (k-n) 

1 n 1 n 

y (k)  = x (k) +n (k) 
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The  process  may  also  be  represented  by  the  pulse  transfer 


function 


G(q  1) 


x(q) 

U(q) 


A (q_1) 


(8) 


i.  -1 . , , -n 

b^q  + . . . . +bnq 

1+a^q  ^"+.  . . +a^q 


where  q is  the  shift  operator  defined  by 


qy (k)  = y (k+1) 


(9) 


Note : All  expressions  used  hereafter  could  be  easily 

generalized  for  the  number  of  input  parameters  b^  [m  , in 
general]  ^ number  of  state  parameters  a^  [n  , in  general]  , 
but  for  ease  of  notation,  we  will  take  m = n. 


Assumptions : 

(i)  Order  n is  known 

(ii)  Residuals  e (k)  are  uncorrelated 
where  e (k)  = generalized  error 

= y (k) 4a^y (k-1) + . . . . +anY (k-n) 
-b^u (k-1) -• • • . -bnu (K-n) 

= yOO-yM(k) 

= y (k) -tp  (k)  £ 


(10-a) 


where  y^(k)  is  the  prediction  of  the  model  based  on  process 
observations  y (k-n) , . . . . ,y (k-1) 


\i>(k)  - [-y  (k-1) . . . .-y (k-n) . . .u(k-l) . . . .u(k-n)  ] 


6 = [a, 


a . • « b ..  • . • » b ] 
n l n 


Minimizing  the  loss  function 


N+n  0 

V = l e2(k) 
k=n 


(10-b) 


and  using  the  notation 


y 1 = [y(n)y(n+l) y(n+N)] 


I -y (n-l) -y (n-2) — — y ( 0 ) 
I =|  -y  (n)  -y(l) 


u (n-l) u (n-2) ... u (0) 


-y  (n+N-1) -y  (N)  . u (n+N-1) 


u(N2J 


we  note  that  e is  linear  in  ai  and  b^. 

Thus  V is  minimized  by 

6 = ~1'l'Ty  if  TT4'  is  not  singular. 

Thus  the  least  squares  estimate  is  simply  calculated  from 
a set  of  observations  y and  a set  of  inputs  u. 


Source (s)  of  error: 

(i)  If  assumed  order  is  wrong,  there  can  be  considerable 
error.  So  test  for  model  order 

(ii)  Principal  problem  is  of  correlated  e(k),  as  is  quite 
usually  the  case. 


H . * • ' « • •<  « 


3ls  can  still  be  computed  but  suffers  from  bias  as 
can  be  seen  simply  by  examining  The  y elements  of 

P are  really 

y(k)  = x (k)  + n(k)  (13) 

where  n(k)  is  additive  measurement  noise  at  the  process 

2 T 

output.  The  bias  is  caused  by  the  n (k)  terms  in  V P 
and  its  asymptotic  value  is 

E[B-3]  = [E(¥T40]_1  E(TTe)  (14) 

T 

where  3 = [a^. . . .a^. .b^. . • -b^l 

To  deal  with  correlated  residuals  several  techniques 
have  been  suggested  - generalized  least  squares,  maximum 
likelihood,  instrumental  variables. 

B . Generalized  Least  Squares 
(Isermann  et  al  [1974]) 

Clarke  [1967]  tries  to  overcome  the  bias  problem  by 
introducing  filters. 

Process  Model: 

(Fig.  5) 

AM(q_1)y(k)  = BM(q-1)u(k)  + w(k)  (15) 

A A 

or  yka  = ukb  + w(k) 

where  w(k)  are  correlated  random  variables  and  the  other 
quantities  are  defined  as  before. 

(i)  The  first  step  is  obtaining  a L.S.  estimate  from 

A 

(15),  which  result  in  biased  estimates  3^. 


24 


(ii)  Then  the  residuals  are  calculated  and  analyzed  by 
autoregression,  assuming  a model 

w (JO  = -f  1w  (k-1) -f  2w  (k-2)  . . .-fvw  (k-v)  +e  (k)  (16) 

respectively.  That  is 

w(k)  = 5(k)f  + e(k)  (17) 

where  e(k)  are  uncorrelated  random  variables  and 
the  order  v has  to  be  chosen  properly.  Least  squares 
estimation  of  the  filter  parameters  using 


(18) 


w 


1 -w (n-1) . . . . -w (n-v) 
! . 

“fl 

e (n) 

1 

= i 

• 

+ 

• 

t • 

1 

( • 

* 

• 

1 

' -w(n+N-l) . .-w(n-v+N) 

. 

e (n+N) 

W 

f 

e 

leads  to  f = [WTW] -1WTw 


(19) 


(iii)  The  input  and  output  sequences  are  filtered  according 
to 


u(k)  = u,  f + u(k) 


y(k)  = yk  f + y (k) 


(20) 


(iv)  A new  L.S.  fit  is  made  with  these  filtered  u(k) 
and  y(k)  and  new  matrices  ¥. 

(v)  Repeat  from  (ii) . 

This  method  is  not  iterative  in  the  process  input-output 
data;  a whole  sequence  of  observations  is  handled  in  a one-shot 
manner . 
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Assumptions  and  associated  sources  of  error: 

(i)  Order  n is  assumed  known,  hence  the  same  error  as 
was  mentioned  in  L.S.  case  could  occur. 

(ii)  Order  v of  the  filter  has  to  be  chosen  appropriately 
or  6 estimates  will  still  be  biased. 

C.  Maximum  Likelihood 
(Eykhoff  [19743) 

This  is  a widely  used  technique  pioneered  by  Astrom  and 
Bohlin  [1965] . Up-to-date  discussions  may  be  found  in  Astrom 
and  Soderstrom  [1974] , Gupta  and  Mehra  [1974]  and  Kashyap  and 
Nasburg  [1974] . 

Process  Model; 

(Fig.  6) 

Taking  the  process  as 

A(q'1)y(k)  = B (q-1 ) u (k)  + XC (q-1) n (k)  (21) 


where  all  quantities  are  defined  as  before  and 

n(k)  = additive  noise,  independent,  N(0,1) 
X = level  of  noise  signal 

C(q-1)  = 1+c.^q  1+c7q  2 +...+  c^q  n 
one  may  choose  for  the  model 


CM(g_1)e(k)  = AM(q  1)y (k) "BM(q_1)u(k) 


Written  in  vector  form,  (21)  and  (22)  become 
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( 21-a) 
(22-a) 

Because  the  noise  n is  Gaussian,  the  likelihood 
T 

function  for  e = {e  (1) , . . . , e (N) } can  be  found  as: 


ay  = bu  + Ac  n 
ay  = J3u  + ye 


1 - T 

L(e;u,a,B,y,A)  = . exp 

(2tt)n/^An  2XZ 


(23) 


£n  L = 


N 1 

- £n  2"  - N£n  A-  — 


N 


2 A k=l 


I e (k) 


(24) 


The  M.L.  estimate  A follows  from 


N 


l e (k) 


lx 4n  l - x + — 3 -r  - ° 


A I A 
'A- A 


N 


T2  k=l 
or  A = 


1 e (k) 


N 


(25) 


The  likelihood  function  may  be  considered  as  a function 

fp  A A A A A ^ 

0 and  A , where  0 = [a,.... a tb, ...b  Icn...c  ].  The 

1 n . 1 n . 1 n 

logarithm  of  the  likelihood  function  may  be  noted  to  be  linear 

A A A 

in  a^  and  b^  and  non-linear  in  c^.  Consequently,  finding 
the  maximum  of  L or  £n  L by  differentiation  and  equating 
to  zero  is  not  necessarily  a simple  procedure.  One  may  proceed 
as  follows.  First  determine  0 such  that 


V (0)  = l e2 (k) 
k=l 


(26) 


is  minimal  with  respect  to  9. 
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>.  . 


. a . 


(27) 


Then,  X2  = ^ min  V(9) 

The  popularity  of  ML  estimates  stems  principally  from 
their  properties  of  consistency,  asymptotic  efficiency  and 
asymptotic  normality.  It  is  also  possible  to  theoretically 
compute  the  accuracy  of  the  estimates. 

Assumptions  and  sources  of  error: 

(i)  Order  n is  assumed  known  - hence  the  possibility 
of  the  associated  error. 

(ii)  Normal  distribution  is  assumed  for  n(k).  Non-normality 
may  seriously  affect  the  desirable  properties  of  ML 
estimates  (viz.  consistency,  asymptotic  efficiency,  etc.) 

D.  Instrumental  Variabales 
(Isermann  et  al.  [1974] ) 

Generalized  least  squares  and  maximum  likelihood  methods 
use  as  their  noise  model  a filter  driven  by  white  noise.  If 
only  the  process  dynamics  are  of  interest  then  instrumental 
variables  can  be  used  to  deal  with  correlated  residuals.  Refer 
to  Wong  and  Polak  [1967],  Young  [1970],  Finigan  and  Rowe  [1974] 
and  Pandya  [1974]. 

Process  Model: 

y = ^3  + e (23) 

T 

with  e = [e (n)  e(n+l)  e (n+N) ] and  the  other  quantities 

as  in  equation  (11)  for  the  least  squares  case. 


Premultiplying  (28)  by  J so  that 


T T ~ ip 

J y = J ¥6  + J e 


where  J is  called  the  instrumental  matrix  satisfying 


E{JTe}  = 0 


T 

E{ J nonsingular 


the  parameter  estimates  are  then  obtained  as  unbiased 


ones  from 


6 = [J^r1  JTy 


The  elements  of  J are  chosen  to  be  uncorrelated  with 
the  residuals  e. 

Wong  and  Pclak  [1967]  and  Young  [1970]  demonstrated  the 
existence  of  optimal  instrumental  variables  and  they  used 
the  calculated,  undisturbed  output  signal  as  instrumental 
variables,  taking  the  parameter  estimates  as  the  parameters 
of  an  auxiliary  model.  If  h is  the  output  of  the  auxiliary 
model,  the  instrumental  matrix  becomes 


-h(n-l)  -h  (0) .u (n-1) . . . . u(0) 


-h  (n+N-1) 


-h (N) .u (n+N-1)  u(N) 


Assumptions  and  sources  of  error: 

(i)  System  order  n has  to  be  assumed  known  - hence  the 
usual  chance  of  error. 

(ii)  Possibly,  correlation  between  auxiliary  model  parameters 


1 


. *• 


and  e may  not  be  zero,  violating  (30)  and  destroying 
the  unbiasedness  of  the  estimate. 

This  is  a popular  method  which  works  well,  the  drawback 
being  the  added  computation  of  the  instrumental  matrix. 

As  a final  note  on  off-line  techniques,  we  should  mention 
that  the  identification  scheme  developed  by  Tse  and  Weinert 
[1973]  using  the  canonical  form  of  Weinert  and  Anton  [1972] 
does  not  assume  the  order  n of  the  process  under  investiga- 
tion but  estimates  it  from  input/output  data  before  estimating 
the  parameters.  The  process  model  assumed  is 

x(k+l)  = Ax(k)  + Bv  (k) 
y (k)  = Cx  (k)  + v (k) 

where  A,  B,  C are  unknown  matrices,  and  so  are  n (the 
process  order)  and  Q , the  covariance  of  the  zero  mean 
Gaussian  process  (v(k)}. 
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Chapter  IV  - On-line  Techniques  for  Identification 

We  now  consider  identification  technqiues  with  the  widest 
application,  viz.  on-line  schemes.  As  we  stated  earlier, 
several  of  the  off-line  technqiues  can  be  put  into  iterative 
form  (with  respect  to  new  measurements)  for  on-line  implemen- 
tation . 

A.  On-line  Least  Squares 

Process  model  and  assumptions: 

These  are  identical  to  the  off-line  case.  The  recursive 
least  squares  estimate  is  obtained  by  writing  equation  (12) 


6 


LS 


in  partitioned  form  and  introducing  the  matrix  inversion 
lemma,  Friedman  [1954]: 

6 (k+1)  = 6(k)  + [^(k+l)P(k)^T(k+l)+l]_1  • 

(34) 

•P  (k)^,T(k+l)  [y  (k+1)-*  (k+1)  B (k)  ] 
with  P (k+1 ) = P (k)  [ I-^T  (k+1)  u,  (k+l)P  (k)  • 


(35) 

• [^{k+l)P(k)pT(k+l)+l]-1] 

P = [VT+]_1.  (36) 


Sources  of  error: 

Same  as  for  the  off-line  case.  Added  problem  of  choosing 
P ( 0 ) and  6(0)  which  may  be  taken  as  0. 
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B.  On-line  Generalized  Least  Squares 
(Isermann  et.  al  [1974]) 

A recursive  generalized  least  squares  algorithm  was 
developed  by  Hastings-James  and  Sage  [1969] . Using  the  same 
process  model  and  assumptions,  the  recursive  equations  are 
set  up  as  in  (34)  and  (35): 


6 (k+1) 


P (k+1) 


6 (k)  + [i|f  (k+l)P(k)^T(k+l)+l]"1* 

•P(k)^T(k+l)  [y  (k+1)  -'P  (k+1)  3 (k)  ] 

P (k)  [I-^T  (k+1)  ']>  (k+1)  P (k)  • 

• [’Jj  (k+1)  P (k)  (k+1)  +1]  _1] 


(37) 


(38) 


f (k+1)  = f (k)  + [?(k+l)Q(k)CT(k+l)+l]_1- 

■Q(k)  CT(k+l)  [w  (k+1) -C  (k+1)  f (k)  ] 


(39) 


Q (k+1)  = Q(k) [l-CT(k+l) 5 (k+l)Q(k) * 


(40) 


[? (k+1) Q (k) CT (k+1) +1]-1] 

Initial  matrices  ?(0)  and  Q(0)  can  be  chosen  as 
diagonal  matrices  with  elements  as  large  as  possible  without 
creating  instability.  The  initial  3(0)  can  be  zero.  An 
exponential  weighting  of  past  data  using  a weighting  factor 
p (Hastings-James  and  Sage  [1969]),  in  the  terms 


[T  (k+1)  P (k)  'fT  (k+1)  +p  ] of  eqs.  (37)  , (38) 
P (k+1)  =ip(k)[I  - ] eq.  (38) 


and  in  the  analogous  terms  of  (39)  and  (40),  prevents  the  first 
estimates  from  becoming  too  poor,  thus  improving  the  convergence. 
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c. 


On-line  Instrumental  Variables 


Process  model  and  assumptions: 

These  are  the  same  as  for  the  off-line  method.  As  in 
equations  (34)  and  (35) , one  may  write  the  recursive  form 
for  on-line  implementation: 


with 


6 (k+1 ) 


P (k+1) 


P (k) 


B(k)  + PP  (k+l)P  (k)  jT(k+l)+l]_1- 

• P (k) jT (k+1) [y (k+1) (k+1) § (k) ] 

P (k) [I-jT(k+l) j (k+l)P(k) • 

• PP  (k+l)P  (k)  jT(k+l)+l]_1] 

[JT(kmk)  r1 


(41) 


(42) 


(43) 


j (k)  = [-h(k-l)  ....  -h (k-n) | u (k-1) . . . . u (K-n) ] 

(44) 


Young  [1972]  introduced  a time  delay  and  a low  pass 
filter  (Fig.  7)  before  updating  the  auxiliary  model,  to  ensure 
that  the  auxiliary  model  parameters  are  not  correlated  with 
e at  the  same  instant  and  to  smooth  the  estimates.  One  may 
use  a low  pass  filter  8_,,„(k)  = ( 1-y ) 6 (k-1)  +yf$  (k)  where 

y is  small  (y^  0.03). 

Sources  of  error: 

In  addition  to  the  possibilities  of  error  mentioned 
for  the  off-line  method,  choice  of  y influences  the  algorithm. 
P(0)  has  to  be  chosen,  and  it  may  be  taken  as  a diagonal 

A ^ 

matrix  with  large  elements.  Initial  values  of  6 and  B 

flUX 

can  be  zero. 
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D. 


Stochastic  Approximation 
(I sermann  et.  ai  [1974] 


This  is  one  of  the  most  popular  methods  currently  in  use 
for  sequential  estimation  of  process  parameters.  It  was  first 
introduced  by  Robbins  and  Monro  [1951],  generalized  by  Dvoretsky 
[1956]  and  treated  extensively  by  Albert  and  Gardner  [1967].  Its 
main  characteristic  is  the  simplicity  of  its  implementation  which 
makes  it  attractive.  An  up-to-date  survey  is  Saridis  [1974  a] . 

Process  model: 

The  same  model  as  in  the  least  squares  case  is  used,  i.e. 
single  input-single  output  description  as  in  (10-a) 

y (k) +a^y (k-1) +. . . .+any (k-n)  = b^u (k-1 ) . . . +bnu (k-n) +e (k) 


Consider  the  estimation  of  the  impulse  response,  g(v)  , of 
this  process.  The  following  algorithm  may  be  used  (to  illus- 
trate stochastic  approximation) . 


G ( k+  £ ) = G(k-l)+y  (k)U(k+£)  • [y  (k+1) 

-UT (k+£) G (k-1) ] 

fp  /*.  /N  A 

w i th  G = [9 ^ 1 9 2 9 • * • r 9 ^ 1 

UT(k)  = [u  (k-1)  u (k-2) u (k-JL)  ] 

VOO  - t(5)  - 5^7  : «lk)  - ITT 

k = 1 , £.+2 , 2 £ + 3 , 3£f  4 , . . . . 


(45) 

(46) 

(47) 

(48) 


If  there  are  2n  unknown  parameters  of  0 , £ = 2n  values 
of  the  g^  are  estimated  by  equations  (45)  and  the  parameters 

A A 

a.  and  b.  are  calculated  as  follows. 

1 1 


Since  the  impulse  response  g(v)  has  been  estimated, 
one  can  estimate  the  response  y using 

CO 

y (t ) = l g (t-v)  u ( v)  (49) 

v=0 

Then,  using  (6) , one  can  write 


ry  ( l ) 

"o' 

0 

u (0)  0 

r ~ 
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■:yU) 

i _j 
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y 

-y (Z-l)-y {1-2) 

-y (*-n) 
R 

u(*-l) 

u (£-n) 

A i 

i b 
L_  nj 

B 

(50) 

/V  A A A 

One  then  estimates  (a1 , . . . , an,b^ , . . . ,b  ) by  a least 
squares  computation 

8 = [Rk]  (51) 

Equations  (49) -(51)  become  especially  simple  for  a step 
input  used  for  the  identification,  i.e.  u(0)  = 1,  u(l) 

= u(2)  = u(3)  ...  = 1.  This  smooths  the  noise  present  in 
g(~)  because  of  equation  (49). 

The  algorithm  (Stochastic  approximation)  of  eq.  (45) 
converges  in  the  mean-square  sense  to  the  true  parameter 
values  under  the  following 
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Assumptions : 


(i)  The  input  u(k)  is  an  independent  random  variable 
with  E{u (k) } = 0. 

(ii)  The  noise  n(k)  is  an  independent  random  variable 
with  E{n  (k) } = 0 . 

(iii)  As  usual,  system  order  n is  assumed. 

Sources  of  error: 

It  is  evident  from  the  assumptions  that  violation  of 
any  of  them  could  cause  problems  in  the  estimation  scheme  - in 
accuracy  or  in  convergence  or  in  both. 

E.  Correlation-Cum-Least  Squares 
(Isermann  et.  at.  [1976] ) 

Conceptually  this  is  one  of  the  simplest  methods  in  use. 

The  estimation  scheme  determines  correlation  functions  and 
then  estimates  the  parameters  of  the  desired  parametric  model 
by  the  method  of  least  squares.  Correlation  techniques  have  been 
studied  extensively  and  one  might  mention  Buchta  [1969] , Hayashi 
[1969],  Reid  [1969  a&b] , Stassen  [1969]  and  Gerdin  [1970]  as 
representative  of  a large  body  of  literature  in  the  area. 

Process  model: 

As  in  equation  (10a) 

y(k)  = -a1y(k-l)  ...  -any (k-n) +£1u (k-1) + . . .+bnu (k-n) +e (k) 
Assumptions : 

Input  and  output  signals  are  stationary  random  variables. 
The  autocorrelation  function  (acf)  of  the  input  then 
may  be  written  as 
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Sources  or  error: 


Non-stationarity  of  input  and  output  signals  invalidate 
the  scheme  outlined.  White  noise  is  not  an  absolute  require- 
ment, it  simply  facilitates  the  computation  a great  deal. 

I , the  number  of  impulse  response  values,  needs  to  be  chosen 


Chapter  V - Concluding  Remarks 


No  discussion  of  identification  techniques  would  be 
complete  without  a comparative  evaluation.  Surprisingly 
enough,  comparisons  of  different  identification  methods  are 
rare  in  the  literature  - Isermann  et.  al  [1974],  Saridis 
[1974b] , Gustavsson  [1972]  are  recent  examples  of  comparing 
on-line  schemes.  With  regard  to  the  on-line  techniques 
that  have  been  described  in  this  paper,  we  could  make  certain 
observations.  For  general  linear  processes,  correlation 
methods  show  most  advantages  compared  to  the  other  methods, 
viz.  instrumental  variables,  stochastic  approximation,  gener- 
alized least  squares  and  least  squares  (Isermann  et.  al  [1974]) 

Very  good  performance,  shortest  computation  time,  100 
percent  overall  reliability  with  no  problems  of  poor  conver- 
gence or  instability,  choosing  only  one  factor  a priori  (the 
number  of  impulse  response  values  desired)  - all  these 
characterize  the  correlation  techniques.  As  mentioned  before, 
while  the  method  is  not  restricted  to  white  noise  inputs,  the 
computational  expense  is  smallest  with  such  inputs. 

Instrumental  variable  methods  perform  well  for  most 
processes,  almost  as  good  as  the  correlations  technique.  The 
principal  shorcoming  is  the  choice  of  a filter  factor,  which 
at  times  could  be  crucial. 

Stochastic  approximation  methods  enjoy  the  advantages 
of  short  computation  time  and  easy  implementation.  The  draw- 
back lies  principally  in  the  choice  of  the  gain  factors  - 
there  is  no  general  rule  to  use. 
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Generalized  least  squares  methods  perform  poorly 
compared  to  the  above  methods,  but  better  than  L.S.  methods, 


which,  however,  take  less  time  than  the  G.L.S.  method. 

Several  questions  remain  unanswered.  We  shall  raise 
only  a few  of  these  here.  For  on-line  techniques,  can  some 
method  be  devised  to  insure  better  starting  estimates?  How 
should  gain  factors  be  chosen  to  improve  rate  of  covergence? 
How  should  one  deal  with  the  general  multivariable  problem 
(the  methods  discussed  were  exclusively  for  single  input 
single  output  models)?  Again,  the  nature  of  the  disturbances 
at  the  output  of  a process  will  have  a direct  effect  on  the 
performance  of  any  identification  technique.  It  seems  reason- 
able to  address  the  problem  of  trying  to  maintain  almost 
uniform  performance  against  a wide  variety  of  output  perturb- 
ances. We  have  already  started  investigations  in  this  area 
in  the  hope  of  being  able  to  devise  some  robust  procedures 
for  identification. 
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Chapter  I - Introduction  And  Scope  Of  The  Report 

The  problem  of  system  identification,  as  control 
engineers  call  it,  is  also  extensively  studied  by  statis- 
ticians under  the  name  of  time  series  analysis.  Consider 
data  {zt,t  = 1,2,3,...}  - in  business,  economics,  engineering 
and  natural  sciences  - which  occur  in  the  form  of  time  series 
where  observations  are  dependent  and  where  the  nature  of 
this  dependence  is  of  interest.  The  techniques  available 
and  utilized  for  the  analysis  of  such  series  of  dependent 
observations . are  called  time  series  analysis.  That  is  to  say, 
given  observed  data,  time  series  analysis  is  concerned  with 
inference  from  what  was  observed  to  what  might  have  been 
observed.  In  general,  the  aims  of  time  series  analysis  are 

(i)  to  understand  the  process  generating  the  time  series. 

(ii)  to  predict  the  behavior  of  the  time  series  in  the 
future. 

Quantitatively,  to  simulate  and  predict  a time  series 
{z,.}  , and  to  understand  the  generating  mechanism,  one  models 
it  as  the  output  of  a dynamic  system  whose  input  is  white 
noise.  Obviously,  one  may  thus  consider  time  series  analysis 
as  identification  of  systems  with  white  noise  inputs.  Thus, 
several  methods  of  time  series  analysis  are  applicable  to 
identification  problems  and  vice  versa.  The  task  is  divided 
into  three  parts: 
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(i)  Postulating  a model  (probabilistic)  for  the  process 
under  investigation  in  which  some  parameters  are 
unknown . 

(ii)  Estimating  numerical  values  of  parameters  for  the 
hypothesized  structure. 

(iii)  Diagnostic  checking  for  the  adequacy  of  the  hypothe- 
sized model. 

. 

The  discussion  here  will  be  restricted  to  the  principal 
methods  for  analyzing  stationary  time  series;  that  is,  a partic- 
ular version  of  task  (ii) . These  techniques  can  be  divided 
into  two  classes: 

a)  Frequency  domain  approach:  Mainly,  spectral  analysis  . 

b)  Time  domain  approach:  Estimating  parameters  of  a 

hypothesized  representation  (model) . 

The  subsequent  material  is  based  on  Parzen  [1961,  1967  and 
1974],  Jenkins  [1961]  and  Box  and  Jenkins  [1970]. 

With  regard  to  parameter  estimation,  there  are  many 
methods  that  are  used  by  statisticians.  Several  of  these 
techniques  are  familiar  to  systems  engineers-  e.g.  ordinary 
and  generalized  least  squares,  instrumental  variables, 
maximum  likelihood  (for  a discussion  of  these  and  others,  see, 
for  instance,  Kashyap  and  Nasburg  [1974]).  The  focus  here 
will  be  on  correlation  methods  used  in  time  series  analysis. 
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Chapter  II  - Frequency  Domain  Approach:  Spectral  Analysis 
A.  Introduction 

Roughly  speaking,  spectral  analysis  is  concerned  with  a 
study  of  {zt}  from  the  viewpoint  of  its  frequency  content. 

It  has  long  been  traditional  among  physical  scientists  to 
consider  a time  series  as  a phenomenon  caused  by  the  super- 
position of  sinusoidal  waves  of  various  amplitudes,  frequencies 
and  phases.  The  notion  of  the  spectrum  is  a central  one  in 
the  analysis  of  time  series.  Spectral  (harmonic)  analysis 
deals  with  the  theory  of  decomposition  of  a time  series  into 
sinusoidal  components. 

One  notes  that  for  many  time  functions  x(t)  , such  a 
decompos  ition  is  provided  by  the  Fourier  transform 

OO 

s(u)  =2 — / e ltw  x (t ) dt  (1) 

— 00 

However,  no  meaning  can  be  attached  to  this  integral 
for  many  stochastic  processes  (x(t)}  since  their  sample 
functions  are  non-periodic  undamped  functions  and  therefore 
do  not  belong  to  the  class  of  functions  usually  considered 
by  the  theories  of  Fourier  series  and  Fourier  integrals.  On 
the  other  hand,  it  is  reasonable  and  possible  to  define  a 
concept  of  harmonic  analysis  of  stochastic  processes  - that 
is,  a method  of  assigning  to  each  frequency  u>  a measure 
of  its  contribution  to  the  'content'  of  the  process  - as  was 
first  demonstrated  by  Weiner  [1930]  and  Khintchine  [1933]. 


S3 


A time  series  may  be  represented  as  a superposition  of 
sinusoidal  wave  forms  with  'independent  amplitudes'  if  and 
only  if  it  is  stationary. 

A discrete  parameter  time  series  {z^.t  = 0,+l,+2...} 

(or  a continuous  parameter  times  series  (z  ,-®  < t < °°})  is 
said  to  be  (wide-sense)  stationary  if  the  covariance  function 


is  a function  only  of  v.  Here  it  is  assumed  that  E[zfc]  = 0 / 
but  all  results  can  be  extended  to  the  case  where  E [z^]  ? 0. 

OO 

If  it  is  assumed  that  £ | R (v) | < » , then  the 

V=— co 

spectral  density 


f(aj)  = E e“lvuJ  R ( v)  -it  < u < ir 

J v=— oo 

exists  and  provides  a spectral  representation 

R(v)  = / e1VaJ  f(w)da)  v = 0 , + 1 , +_2 , . . . 

-IT 


(3) 


(4) 


Estimation  of  f , and  hence,  the  determination  of  the 
frequencies  oj  at  which  f has  local  maxima,  is  the  principal 
concern  of  spectral  analysis. 

B.  Estimation  of  Soectral  Density  f (oj) 

> 

Given  a time  series  sample  of  size  T (z^t  = 1,2,  ..,T}  , 
the  sample  covariance  function  (v)  and  the  sample  spectral 
density  function  (or  periodogram)  fT(w)  are  defined  by 


i T; I v I 

rt(v)  = t 2t  zt+|v]  v = °'±1 ±(T-1} 


(5) 


= 0 


v = +T,+ (T+l ) , 


f . , 1 | r -itu  I 2 

fT(u))  = t 1 e Zt|  -7T  < O)  < IT 

t — 1 


(6) 


It  may  be  shown  that  fT(to)  and  H,p(v)  are  Fourier 
transforms : 


fT<“>  = 2?  ^ e iV%(v)  (7) 

v=-T 

R^v)  = /TTe1Va,fT(o))du  v = 0 , + 1 , . . . (8) 


However,  there  is  a difficult  problem  in  empirical 
spectral  analysis  of  a stationary  time  series  possessing  a 
spectral  density.  This  stems  from  the  fact  that  the  obvious 
estimate  of  f (to)  , namely  fT(to)  just  defined,  is  not  a 
consistent  estimate  of  f(io)  although  R,p(v)  i_s  a consistent 
estimate  of  R(v)  at  each  v. 

That  is,  it  may  be  shown  that 


lim  E [ | R (v)-R(v) |2]  =0 

'P-VOO 

(9) 

P [lim  R (v)  = R (v)  ] = 1 

»r-*oo 


On  the  other  hand,  it  may  be  shown  that 
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lim  E [elufT  (w)  ] = (1-iuf  (to)  )_1  (10) 

for  every  real  u and  frequency  to. 

Consequently,  for  every  real  number  x , 

lim  P[f  (to)  > x]  = e~x/f(w)  (11) 

which  means  that  f T ( u> ) is  exponentially  distributed  with 
mean  f(oj).  So,  unless  f(<o)  = 0 , there  is  no  mode  of 
probabilistic  convergence  in  which  fT(co)  f (to)  as  T-”00. 
However,  it  is  not  difficult  to  construct  sequences  of  estimates 
of  f (to)  which  are  consistent. 

Since  R^ (v)  is  a consistent  estimate  of  R(v)  , i.e. 

/ e fj  (to) do  -*■  j e f(<o)dto  (12) 


it  follows  (Parzen  [1957  a])  that  for  every  bounded  continuous 
function  A (to)  , 

CO  CO 

/ A(co)f_,(to)dio-*-/A(u))f((jo)dio  (13) 

— oo  — oo 

The  convergence  in  (12)  ana  (13). is  the  same  mode  as  in  (9). 

A great  deal  of  effort  has  been  expended  by  researchers 
in  constructing  consistent  estimates  of  f(w)  , and  Grenandar 
and  Rosenblatt  [1957]  have  shown  that  one  need  only  consider 
estimates  of  the  form 
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(14) 


= 2?  4 e"1V“0  k.ptv)  Rt(v) 


v=-T 


where  the  constants  kT (v)  are  to  be  chosen  as  even  functions 
of  v.  These  estimates  may  also  be  written  as  sample 
spectral  averages 

v 

f 

-7T 


fT(a>0>  = f KT(^-w0)fT(w)doo 


(15) 


where  the  spectral  window 


KT(w)  = 2r  I eivukrT,(w) 


(16) 


v=- rJ 


10 


It  is  assumed  that  K^tw)  achieves  its  maximum  at 
0.  Its  bandwidth  may  then  be  written  as 
m 


3(kt) 


= / K U)du 


Kt(3) 


(17) 


* 

The  bandwidth  of  the  estimate  fT(io)  is  the  bandwidth 
of  its  spectral  window. 

★ 

In  order  to  specify  an  estimate  fT(io)  , one  must  state 
the  covariance  averaging  kernel  (also  called  lag  w’indow) 
kT(v).  There  are  mainly  two  methods  for  generating  kT(v)  , 
which  include  as  special  cases  most  of  the  estimates  suggested 
by  various  authors. 

Let  h(u)  be  a bounded,  even,  square  integrable  function, 
defined  for  all  real  u such  that 
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1 1 - h (u)  | / | u ( 


is  a bounded  function  of  u. 

One  class  of  estimates,  called  the  algebraic  type 
(Parzen  [1961]),  has  f*(ui)  defined  by  (14)  with 

JcT(v)  = hfv/Hj.)  (18) 

where  the  are  positive  constants  tending  to  0 as 

T-*-  « in  such  a way  that  (M^,/T)  ■+■  0. 

The  other  class  of  estimates,  called  the  exponential 
type  (Parzen  [1961]),  has  f*(cu)  defined  by  (14)  with 

kT (v)  = h(ATeaf V! ) (19) 

where  AT  are  positive  constants  -*-0  as  T -*■  00  in  such 
a way  that  (log  A^J/T-^O  and  a is  a positive  constant. 

The  following  is  a list  of  different  h(u)  corresponding 
to  estimates  used  by  various  authors. 

(Bartlett  [1950]) 

(20) 

M < 1 

(Blackman  & Tukey[1959]) 

M > i 

(21) 

(called  truncated  periodogram) 


h(u)  =1-  1 u | | u 1 < 1 

= 0 juj>  1 

h(u)  = 1/2  (1+COS  7T  u) 
= 0 

h (u)  =1  | u j <1 

|u|  > 1 


0 
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(22) 


h (u)  = 


sir,  u 
u 


(usually  attributed  to  Daniell) 
(Parzen  [1961] ) 


(23) 


h(u)  = 1- | u | " | u | > 1 (Parzen  [1957b]) 

= 0 otherwise 

for  some  constant  g > 1 to  be  determined 


(24) 


h (u)  = l-6u2+6 | u | 3 1 u | < 1/2 

= 2 (1- | u j ) 3 1/2  < | u | < 1 

= C otherwise 


(Parzen  [1957c] ) 

(25) 


Other  possible  choices  for  kT(v)  are  given  in  Parzen 
[1957  d,  1958] . 

★ 

An  estimate  f^tui)  is  said  to  be  of  non-negative  type 

* . 
if  fT(u>)  > 0 Vo).  A necessary  and  sufficient  condition 

for  this  is  > 0 Vco. 

Also,  assume  that  the  kernel  h(u)  satisfies 

| 1— h (u) | < h |u|g  Vu  (26) 

for  some  exponent  q > 0 and  constant  h^. 

The  largest  real  number  q such  that  kernel  h(u) 
satisfies  (26)  for  some  finite  hg  is  called  the  characteristic 
exponent. 

★ , 

An  estimate  fT(o)  is  said  to  be  of  truncated  type  if 

there  exists  a real  number  ^ < T such  that 

kT(v)  = 0 for  | v ! > itVp  (27) 
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If  there  exists  a smallest  real  number  mT  satisfying 

(27) , we  call  it  the  truncation  point  of  the  estimate.  The 

advantages  of  truncated  estimates  are  reduction  in  computation 

since  all  values  of  R„,(v)  do  not  have  to  be  computed. 

▲ 

A kernel  h(u)  satisfying 


h(u)  > 0 for  |uj  < 1 

h(u)  = 0 for  I u I > 1 


(23) 


gives  rise  to  algebraic  estimates  with  truncation  point 

nVp  = iip  (28a) 

and  to  exponential  estimates  with  truncation  point 

mT  = - £ log  At  (28b) 

Except  for  (23),  all  kernels  h(u)  listed  satisfy  (28). 
itij  , as  defined  above  in  (23a)  and  (23b),  is  designated 
as  the  truncation  point  of  the  estimate  even  if  the  estimate 
is  not  of  the  truncated  type.  The  statistical  properties  of 
estimates  are  best  expressed  in  terms  of  n^. 

Parzen  [1961]  advocates  the  choice  of  kernel  on  the 
basis  of  the  following: 

(i)  Bandwidth  and  variance  are  inversely  proportional 

★ 

for  any  kernel.  The  variance  of  the  estimate  fT(w) 
for  a given  bandwidth  is  lowest  for  kernels  (20) , 

(25)  and  (21). 

(ii)  Mean  square  error  criteria  indicate  the  preferability 

of  a kernel  with  characteristic  exponent  q=2 . (23), 

(25)  and  (21)  meet  this  requirement. 
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, Vs. 


. >«  « 


r 


(iii)  Given  a truncated  h(u)  and  truncation  point 

the  variance  of  f ( --o ) is  low  for  (25)  and  (21)  . 
He  thus  concludes  that  (21)  and  (25)  are  the  best 
competitors  for  the  choice  of  kernel  to  use  in 
estimating  f (w) . 


Chapter  III  - Time  Domain  Approach:  Parameter  Estimation 


A.  Introduction 

We  consider  the  time  series  as  the  output  of  linear, 
time  invariant,  discrete  time  systems  subject  to  random 
shocks  (white  noise) . One  can  describe  (parametrize)  such  a 
model  in  several  ways;  but  to  use  the  fewest  number  of 
parameters  ('parsimonious'  parametrization)  one  employs 
a mixed  autoregressive-moving  average  (ARMA(p,q))  representation 


Val2':-l+a2Zt-2  aP2t-P 


- ut+61ut_1  + +SqUt-q 


(29) 


or,  in  operator  notation, 


g (L) z t = h (L)ufc 


(30) 


where  L is  the  lag  (or  backward  shift)  operator 


Lz , = z 


t-1 


g(x)  = l+a,x  +....+  a xb 


h (x) 
2 


= l+3i x +....+  8 x 


(31) 


a = 


E ( | ufc | ] is  the  variance  of  the  white  noise  u^. 


For  this  model  to  represent  a stationary  time  series  the 
roots  of  the  characteristic  equation  g(x)  = 0 must  lie 
outside  the  unit  circle  in  the  complex  plane.  An  ARMA ( p , q ) 
model  for  a stationary  time  series  has  parameters 
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2 

f 8*.  / • • • r 6-/ o . (Again,  zero  mean  is  assumed;  however, 

-L  P 

that  could  easily  be  estimated  as  another  parameter  y) . One 
can  see  that  the  ARMA(p,q)  model  is  just  the  difference 
equation  representation  of  the  familiar  single-input-single- 
output  (SISO)  state  model: 


r “1 

B = | 

i = Ax,  +Bu,  A = 0 I j 

k+1  k k 

. 

a. ....  a i 
1 n 

i J 

’ ' 
!bnJ 

c = [1  0 0....0] 


where  n = p = q. 

The  two  fundamental  processes  - autoregressive  and 
moving  average  schemes  - are  defined  as  follows: 


Autoregressive  (AR(p)) 

The  time  series  {zt}  is  assumed  to  be  generated  as  a 
linear  function  of  its  past  values  plus  a random  shock;  for 
some  integer  p (called  the  order  of  the  AR  scheme) , and 
constants  , 


zt  = aizt-i  +--‘-+  Vt-p+ut 

in  which  the  sequence  {ut}  consists  of 
cally  distributed  (usually  assumed  normal) 


independent  identi- 
random  variables. 


Moving  Average  ( MA ( q ) ) 

The  sequence  { z } is  assumed  to  be  generated  as  a 
finite  moving  (and  weighted)  average  of  a sequence  of  inde- 
pendent, identically  distributed  random  variables  {ut}  < 
for  some  integer  q , (called  the  order  of  the  MA  scheme) , 
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. v„ 


and  constants  , 


2t  = ut+SlUt-l+-”-+8qUt-q 

As  stated  earlier,  we  shall  focus  our  attention  on 

correlation  methods  for  estimating  the  parameters 

2 

B.  Correlation  Methods  for  ARMA  Models 

Let  us  consider  a (wide-sense)  stationary  time  series 

2 

{zfc,t  = 1 , . . . N } with  non-zero  mean  u and  variance  a . 
Since  the  probability  distribution  is  the  same  for  all  times 
t , one  estimates  the  mean  from  a set  (zt,t  = 1,....,N}  of 
observations  by 

N 


= | l zt 

w t=l 


and  the  variance  a by 

z 


%2  1 r , - . 2 

°z  ■ N ,2t-2) 


The  autocovariance  at  lag  k is  defined  as 


Yk  » Cov[zt,zt+k]  = E[(zt-u>  (zt+k-u)l 


(32) 


The  autocorrelation  at  lag  k is 


°k  = EK2t‘u)  (2t+k“u)] 


Se  ((zt~u ) 2 1 e [ (zt+k-y)  21 


= Et(zt~u)  (zt+k_u)] 
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(33) 


1 


■4  • > 


F 


2 

since,  for  a stationary  process,  the  variance  °z = Yq  is 
the  same  at  time  t+k  as  at  time  t. 

Thus  the  autocorrelation  at  lag  k is 


(34) 


A number  of  estimates  of  the  autocorrelation  function 
have  been  suggested  and  their  properties  are  discussed  in 
particular  by  Jenkins  and  Watts  [1968].  It  is  concluded  that 
the  most  satisfactory  estimate  of  the  k^*1  lag  autocorrelation 
pk  is 


r 


k 


(35) 


where 


. N-k 

‘ 5 <zt-5)ut+k-51 

k = 0,1, .. .K 


(36) 


is  the  estimate  of  autocovariance  , and  z is  the  mean 

of  {z  ,t  = 1 N } . K is  taken  > p+q. 

Following  Box  and  Jenkins  [1970],  a procedure  is  now 
given  for  estimating  the  parameters  of  an  ARMA(p,q)  model: 

Zt  = alZt-l+ +apZt-p+eC+Ut~BlUt-l_e2Ut-2 _BqUt-q 


(37) 

p , q are  assumed  known.  {ut}  is  assumed  to  be  a sequence 
of  independent  identically  distributed  (usually  assumed 
normal)  random  variables.  is  the  overall  constant  term. 
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B.l  Mean  and  variance 


N 


- B J,  2t 


(38) 


t=l 


52  = C0 
u 0 


where  cn  is  defined  below 


(39) 


B . 2 Autocovariance  function  (acvf) 


, N-k 

= i l (zt-z) (zt+k-z)  where  k = 1, ,K 

K = no.  of  acvf  values 
desired  > p+q 

(40) 


B.3  Autocorrelation  function  (acf) 


rk  c. 


(41) 


B . 4 


Estimates 


If  p > 0 


(initial)  ctQ  of  AR  parameters 
, solve  the  set  of  p linear  equations 

= x 


where 


C|q+i-j| 


X;  = 


c 


q+i 


i, j = 1#2|.../P 


(42) 


B . 5 Estimates  (initial)  3Q  of  MA  parameters 


a.  Using  the  c^  , the  following  modified  c'  are 
computed : 
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' Jo  Jo 

= c . 

D 


p > 0 (£00  = -1; 


P = O 


(43) 


b.  Then  the  Newton-Raphson  algorithm 


i+1  _i  . 
x = t - h 


(44) 


where  T1h  = f1 


is  used  to  calculate  the  vector  x1+1  at  the  (i+l)st 
iteration  from  its  value  t ^ at  the  it^1  iteration,  where 


t - (t0,t1, . . . ,Tq) 


(45) 


f.  = 
3 


q-3 


Jo 


(46) 


J 


c.  when  | fj | < e , j = 0,1,..., q for  some  pre- 
specified e , the  process  is  considered  to  have  converged 
and  the  parameter  estimates  are  obtained  from  the  final  x 
values  according  to 


Bjo  = - Tj/T0 


j — 


B . 6 Estimate  eQ0  of  overall  constant 


(1  - l ai0 > p > 0 

i=l  u 


p = 0 


B . 7 Estimate  au  of  white  noise  variance 


-2 

a = 
u 


q > 0 


:0  - Sioci  q - 0 


B.8  Backf orecasting  initial  z's 


:(zN-bfi)  *N-b(£)  900  + “iO^-b-i+Ji1  " ^ S jO  [uN-b-j+£^ 


where  = 


^uN-b-j+ 


VbU’i)  1 > 1 


^-b-i+tf. 


I < i 


i > j 


zN-b-j+  l ’ zN-b-j+^-l(1)  1 - j 


1 


V 


with  £ = 1 , 2 , . . . L (lead  time  values).  The  forecasts  are 
obtained  for  each  origin  b = 0,1,... B time  units  before 
the  end  of  the  series.  The  backf orecasts  are  done  up  to  a 
negative  origin  Q'  beyond  which  the  difference  between  z 
and  ^ ( = becomes  negligible. 

B. 9 Calculation  of  residual  sum  of  squares 

a.  Having  backforecast  initial  z's  , the  residuals 
for  a specified  set  of  values  of  the  parameters  are  computed: 

p „ q 

*t  = (2t-^  " 2 GiO(Zt-i~D)  + ^ Bj0*t-j  (54) 

i=l  3=1  J J 

t = Q' ,Q'+  1,....,N  where  Q'  is  as  above  in  B.8. 

b.  For  given  values  of  the  parameters  (y,a,|)  the 
residual  sum  of  squares  is  computed  from 


S(y,a,S)  = l <t>7 

t=Q'  t 


B. 10  Calculation  of  least  squares  estimates 

The  values  of  the  parameters  which  minimize  the  residual 
sum  of  squares  are  obtained  by  a constrained  optimization 
method,  proposed  by  Marquardt  [1963]  and  described  in  the 
following  form  by  Box  and  Jenkins  [1970]: 

a.  Denoting  by  A = ( A^ , X^ , • • • / A^ ) all  the  parameters 
in  the  model,  that  is  A = (p  , starting  values  Afl  are 

specified  along  with  parameters  d and  F which  constrain 
the  search  and  a convergence  parameter  c . During  the  search, 
the  values  <[•  = E[qt|A,z]  and  the  derivatives 


1 


1 


X . 
1ft 


34>t 

7T7 

i 


(56) 


need  to  be  evaluated  at  each  stage  of  the  iterative  process. 

b.  Using  the  residuals,  calculated  as  described  in 
Section  B.9  pt.a(i.e.  according  to  eq.  54),  the  derivatives 
are  obtained  from 


x. 


_ {9t(Xl,0,'*'/Xi,0',,’,Xk,Q)~(i>t(Xl,0'‘'‘,Xi,0*V’',,Xk,0)  ' 


°i 


(57) 


c.  with  <?  ,x.  supplied  from  the  current  parameter 
values,  the  following  computations  are  done: 


(i) 


The  k*k  matrix 


where 


A > {A.j) 


N 

L. . = y x. 

•'i  l * i . 

J t=Q  ' 


x . 

t ],  t 


(ii) 


?he  vector  g with  elements  g1,g2,...,g>  where 


N 


9i  ‘ t lQ- 


’k 


(iii)  The  scaling  quantities  = /A~ 


d.  The  modified  (scaled  and  constrained)  linearized 
equations 
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1 

1 


(58' 


* * * 

Ah  = g 

are  constructed  according  to 


* 

A.  . 

= A. ./D 

±3 

* 

A.  . 

= 1+d 

11 

* 

gi 

= g/D. 

(59) 


The  equations  are  solved  for 
to  give  the  parameter  corrections 


★ 

h , which  is  scaled  back 
hj  , where 


h . 
D 


h*/D . 

y D 


(60) 


The  the  new  parameter  values  are  constructed  from 


X = Xp  + h (61) 

and  the  sum  of  squares  of  residuals  S(X)  is  evaluated. 

e.  If  S (X)  < S ( Xq ) , the  corrections  h are  tested. 

If  all  are  smaller  than  e , convergence  is  assumed.  Otherwise, 
Xq  is  reset  to  the  value  X , d is  reduced  by  a factor  F 
and  computation  returned  to  (c) . 

f.  If  S ( X ) > S(XQ)  , the  constraint  parameter  d is 
reduced  by  a factor  F and  computation  resumed  at  (d) . In 

all  but  exceptional  cases,  a reduced  sum  of  squares  is  eventually 
found.  However,  an  upper  bound  is  placed  on  d , and  if  this 
is  exceeded,  the  search  is  terminated.  When  convergence  has 
occurred,  either  according  to  the  criterion  in  (e) , or  it  is 

71 


1 

I 


* 


assumed  to  have  taken  place  after  a specified  number  of 
iterations,  the  residual  variance  and  the  covariance  matrix 
of  the  estimates  are  calculated  as  follows. 

B. 11  Standard  errors  and  correlation  matrix 

The  estimate  of  the  residual  variance  is  obtained  from 
the  value  of  the  sum  of  the  squares  function  at  convergence 
using 


-2 

au  = 


N-p-q-1 


S (u,a, 3) 


(62) 


and  the  covariance  matrix  V of  the  estimates  from 

V = tVij}  = (xTx)-1  °u  (63) 

where  X is  the  regression  matrix  in  the  linearized  model, 
calculated  at  the  last  iteration  in  the  Marquardt  procedure 
[Eq.  (59)]- 

The  standard  errors  are 

S.  = AT-  i = 1,2,  . . . ,p+q+l  (64) 

i ii 

and  the  elements  of  the  correlation  matrix  are  obtained 

from 


R. . = V. .//V. . v.  . 
ID  ID  11  5D 


(65) 


Finally,  an  estimate  6 Q of  the  overall  constant  term  is 


90  = Mi  -l*L) 


(66) 
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C . Correlation  Estimation  for  Noisy  Measurement  Models 

The  procedures  just  described  are  appropriate  for  analyzing 
time  series  in  which  the  observations  are  assumed  to  be 

uncorrupted  by  noise.  The  only  perturbances  are  the 
random  shocks  u . However,  in  most  cases,  the  observations 
themselves  are  perturbed  by  additive  noise  so  that  one  really 
observes 

yt  = zt  + nt  (67) 

where  {nt>  is  a noise  sequence  independent  of  the  input 
(ut)  and  usually  assumed  white.  It  would  also  be  helpful 
if  {uj  in  general  did  not  have  to  be  white  noise.  The 
method  to  be  described  considers  the  more  general  problem  of 
non-white  (or  'colored')  noise  {nfc}  , the  sequence  (ut) 
also  being  considered  as  a 'colored'  (correlated) 
sequence  of  random  variables.  Of  course,  considerable  simpli- 
fication occurs  if  {ut>  is  a 'white'  sequence. 

We  consider  two  time  series,  {ut,  t = and 

{y  , t = 1, . . . ,N]  corresponding  to  the  input  and  output 
measurements  of  a linear  process. 

Consider  the  representation  of  the  linear  process  to  be 

yt  = 9oV9lYl+--"+gmut-m+nt  (68) 

where  g^.g^,...,  gm  are  impulse  response  weights  for  the 
process.  m is  the  number  of  impulse  response  values  desired 
and  is  at  least  = p+q  , where  p and  q are  the  orders  of 


the  AR  and.  MA  schemes  of  the  ARMA  process  which  one  assumes 
to  have  generated  {u^}.  Larger  values  of  m increase  the 
model  accuracy,  so  the  choice  has  to  be  made  on  the  basis  of 
computational  cost  versus  accracy. 

First,  consider  the  input  sequence  (ut>  to  be  tlle 
output  of  an  ARMA  (p,q)  process  driven  by  white  noise.  That  is, 

d(L)ufc  = h(L)at  (69) 

where  {a^}  a se<3uence  independent  identically  distri- 

buted random  variables. 

2 P 

d(x)  = l+a1x+a2x  t. . .+apxr 

h(x)  = l+S-^x+&2x^'i'’  • • +®qx<^ 


Then  these  . . . ,a  , • • • i Bg  are  computed  by  the 

procedures  described  previously.  Then  the  following  computations 
are  made. 

C. 1 Differencing  and  pre-whitening 

The  input  series  {ufc}  and  the  output  series  {yt)  are 
differenced  to  form  N values  of 


ut  " u t ut 


*t  = yt  ’ yt 


(70) 


where 


u and  yfc  are  the  arithmetic  means  of  the  ufc  and 


y series 


■:& 


■5 


w<  : that  is  to  say,  we  would  like  to  ensure  improved  nprfnrn- 


The  differenced  series  are  then  pre-whitened  to  give 


n ' = N-p  values  of  the  At-Bt  series  according  to 


At  = ut 


Bt  = *t 


t'  ^ 

y a . + y b -A, 

iii  1 ‘-1  j£i  3 ^ 

P <3 

T a . y ' . + V 6 . B . 

ik  jij.  3 t-j 


(71) 


A,  B are  the  means  of  the  At>Bt  series. 

C. 2 Pre-whitened  output  autocorrelation  function 

"l  k(B  -B)(B  -B) 

" Jfl 


rBB  (k) 


l (B.-B)' 
j = l 3 


k — 0 f 1 1 » < • f in 


(72) 


C. 3 Pre-whitened  input-output  cross-correlation  function 

(73) 


where 


r7,tl(k)  = ^—5 

AB  S,ST 


A B 


n '-k 


CAB(k>  ‘ 5 i IBj*k'S)  k * °'1 " 


CAB<-kl  ' CBAlk) 


k = 1 , 2 , . . . ,m 


(74) 


SA  = /=AA<°> 


(75) 


SB  >/CBB  (0) 


C.4  Impulse  response  function  estimate 


C. 5 Noise  variance  and  autocorrelation  function 

Using  the  estimates  g^  of  the  impulse  response  weights, 
the  noise  series  n^  is  regenerated  from 


= yt-g0ut-g1ut_1 


"5nUt-n 


where  n < m 


and  then  the  variance  and  autocorrelation  calculated  as  in 
Sections  B.l,  B.2  and  B.3.  [eqs.  38-41],  from  the  values 
nfc/  t = 1,2,... ,N-n. 

C.  6 Parameter  estimates 

Using  the  impulse  response  weights,  one  may  estimate 
parameters  [a^, . . . ,an,b^, . . . ,b  ] of  the  SISO  state  model 
[eq.  31(a)].  See,  for  instance,  Isermann  et  al.  [1974]. 

D.  Robust  Estimation  of  the  Transition  Parameter  of  an  AR(1? 
Model 


We  conclude  this  chapter  by  considering  the  problem  of 
estimating  a in  the  following  AR(1)  (or  1st  order  Markov) 
model : 


symmetric  about  the  origin.  Stationarity  is  also  assumed, 

which  is  equivalent  to  the  assumption  that  |u|<l. 

The  specific  model  most  used  is  one  for  which  v,  ^ 0 

k 

and  {u^}  are  Gaussian.  Under  such  assumptions  the  estimate 
for  a.  is  the  standard  least  squares  one  given  by 


& 


n-1 


i=l 


Zi2i+1 


Ls,n  n-1  „ 

l z2 

i-1  1 

for  a set  of 


(79) 


observations  z. , 


n 


Unfortunately,  c<Ls  n is  asymptotically  inefficient 
when  {u^}  has  a heavy-tailed  non-Gaussian  distribution  so 
that  { u^, } has  outliers.  Also,  in  the  case  of  non-zero 
additive  effects,  i.e.  v^  / 0 , the  least  squares  estimate 
^Ls,n  is  asymptotically  biased  as  it  converges  to 


a„  = a - a . 


2 2 
1+cT/cT 
x v 


(80) 


where 


x 

2 

3v 


Variance  of  x. 


Variance  of 


This  bias  can  be  considerable  even  when  v^  is  zero  most  of 
the  time:  say  v^  is  distributed  according  to  the  density 

= (l-£ ) 6 (v) + eG (v  1 0 , a2) 

V 

with  c small  (<  0.2) 


f (v) 


(81) 


Thus,  is  zero  1-e  of  the  time  and  comes  from  a 

Gaussian  distribution  e of  the  time. 

Denby  and  Martin  [1975]  and  Martin  and  Jong  [1975]  have 
proposed  and  constructed  robust  estimates  for  a in  the  case 
of  non-Gaussian  u and  non-zero  v by  solving  the  equation 

n-1 

Iig(zi)<Mzi+1-cinzi)  = 0 (82) 

If  u and  v have  finite  variances,  the  estimate  a' 

n 

obtained  above  in  (82)  converges  almost  surely  to  where 

is  the  root  of  the  regression  equation 

m(a')  = -E  g (zi)^  (zi+1-a'zi)  = 0 (83) 

and  is  the  same  sign  as  a and  | | < ] a | . One  can  see 

that  (32)  gives  the  least  squares  estimate  a_  if  g(*) 

and  ip  (’ ) are  taken  as  the  identity  functions . As  in  the 
least  squares  case  - where  ab(=aQ)^a  - it  is  true  that, 
in  general,  for  a particular  g(-)  and  M • ) > • 

However,  use  of  suitable  g(’)  and  -p  ( • ) results  in  a very 
small  bias  when  v^  is  distributed  as  in  (81) , and 
0 < t < 0.2  , so  that  robustness  toward  bias  is  obtained  as 
well  as  robustness  toward  variance.  Martin  and  Jong  [1975] 
may  be  consulted  for  details  of  asymptotic  properties  of 

A 

and  necessary  assumptions  and  conditions. 

Obviously,  there  remains  the  need  to  construct  robust  estim- 
ation procedures  for  the  parameters  of  general  AR(p)  and  ARMA  (p, 
models  with  a similar  non-Gaussian  assumption  on  the  u,  and  v.^0 

»C  K 


Chapter  IV  - Concluding  Remarks 


We  conclude  this  discussion  by  considering  the  relative 
merits  of  frequency  domain  and  time  domain  approaches  to 
time  series  analysis  - specifically,  spectral  analysis  as 
compared  to  correlation  analysis  (Jenkins  [1961]).  For  a 
specific  application,  there  are  principally  two  factors  to 
be  considered  in  choosing  between  correlation  and  spectral 
analyses : 

(i)  The  subsequent  use  of  the  estimated  quantitites. 

(ii)  The  ease  of  physical  interpretation. 

These  considerations  are  best  illustrated  by  specific 
problems : 

a)  Frequency  Response  Studies 


If  the  ultimate  objective  of  the  analysis  is  to  look 
at  the  distribution  of  variance  or  power  with  frequency, 
then  the  spectrum  provides  a direct  answer  to  such  problems. 
Autocorrelations  may  be  used  towards  such  an  end,  but  the 
computations  are  far  more  indirect  and  complicated.  Again, 
in  certain  aspects  of  control  system  design,  the  spectrum 
provides  valuable  information  with  regard  to  the  frequency 
response,  such  as  sharp  peaks  (resonances).  In  general,  the 
spectrum  has  a direct  physical  interpretation  related  to  the 
frequency  response. 

b ) Prediction  and  Simulation 


Since  prediction  and  simulation  are  done  in  time,  it 


is  natural  to  work  in  the  time  domain,  as  variate  values  are 
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required  for  feeding  into  the  prediction  (simulation)  model. 
Spectra  are  of  no  use  in  this  type  of  problem.  Also,  in 
several  control  system  design  problems,  time  domain  models 
(such  as  state  space,  difference  equation)  are  used  extensively. 
Since  parameters  are  directly  estimated  as  functions  of  the 
auto  and  cross  correlations,  it  is  natural  to  work  with  these 
quantities . 

c)  Exploratory  Investigations 

Initial  studies  in  many  fields  are  done  by  building 
models  based  on  a priori  considerations.  Both  correlation 
and  spectral  analysis  may  be  of  considerable  use  in  suggesting 
possible  models,  depending  again  upon  the  type  of  represen- 
tation desired. 

Because  the  autocorrelation  function  and  the  spectrum 
are  transforms  of  each  other,  they  are  mathematically 
equivalent.  However,  as  illustrated  by  the  examples  just 
cited,  the  choice  of  one  or  the  other  will  be  dictated  by 
considerations  of  the  aims  and  goals  of  analyzing  a particular 
time  series. 
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Chapter  1 Problem  Statement  and  Proposed  Solution 
A.  Introduction  and  formulation  of  the  Problem 

We  recall  (from  Sections  1&2)  that  the  on-line 
identification  of  single  input  single  output  linear,  time 
invariant,  noisy  systems  modeled  in  discrete  time  is  of 
considerable  general  interest,  and,  accordingly,  has  been 
studied  most  extensively.  However,  in  the  construction  of 
identification  schemes,  there  remain  interesting  questions 
which  have  yet  to  receive  adequate  consideration  - specifically, 
for  one,  the  modeling  of  the  non-measurable  output  noise. 

It  is  obvious  that  the  nature  of  this  corrupting  noise  has  a 
direct  effect  on  the  performance  of  any  identification  algo- 
rithm. In  general,  though,  there  have  been  somewhat  extreme 
approaches  to  modeling  the  measurement  noise.  At  one  end  of 
the  scale,  the  output  noise  is  assumed  to  be  known  perfectly. 

For  example,  the  maximum  likelihood  method  assumes  Gaussian 
noise  with  known  parameters.  On  the  other  hand,  there  are 
methods  which  assume  (or  require)  little  knowledge  of  the 
measurement  noise  - correlation  analysis,  for  example,  only 
requires  stationarity  of  the  input  sequence  and  the  output 
noise,  and  that  they  be  mutu tally  orthogonal  or  uncorrelated. 
Since  one  can  be  expected,  in  reality,  to  have  some,  if  not 
complete,  knowledge  of  the  operating  environment,  it  seems 
reasonable  to  utilize  a model  in  which  the  output  noise  is 
neither  completely  specified  (as  a Gaussian  process,  for 
example) , nor  left  completely  unspecified.  We  thus  formulate 
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the  following  identification  problem. 

Consider  the  on-line  identification  of  single  input 
single  output  linear,  time  invariant,  noisy  systems  modeled 
in  discrete  time 

xk  = Axk-1  + buk-l 

with  observations  (1) 


under  the  assumptions  that 

(i)  the  order  of  the  system,  n,  is  known; 

(ii)  the  system  is  completely  controllable  and 
observable,  and  stable; 

(iii)  the  distribution  of  the  measurement  noise  w is 

given  by  the  mixture  F (w)  = (1-e)  K(w)  + e C(w),  (la) 
where  K ( * ) is  known  (i.e.  completely  specified)  and  C belongs 
to  some  broad  class  of  distributions.  This  is  to  say,  w^ 
comes  (1-e)  of  the  time  from  a known  distribution  and  e of  the 

36 


time  from  a distribution  C e 0.  For  the  rest  of  the  report, 
we  use  the  following  equivalent  representation  of  (1) : (see, 

for  example,  Luders  & Narendra  [1973]) 


yk 


= -a 


1 yk-l  a2  yk-2  ~a3  yk-3 


“an  yk-r 


Zk  = 


+bl  Uk-1  + b2  Uk- 2 + 


yk  + Wk 


• • +bn  uk-n 


(2) 


where  a2_,...,an,  b^,...,b  are  the  unknown  parameters  to  be 
identified. 

Some  investigations  of  this  problem,  in  somewhat 
different  formulations,  have  been  initiated  by  Nasburg  and 
Kashyap  [1975],  Denby  and  Martin  [1975]  and  Martin  and  Jong  [1975] 
B.  Review'  of  Cross-correlations  method  - possible  shortcomings 

Referring  again  to  Section  1,  one  recalls  that  the 
method  of  cross  correlations  seems  to  be  most  advantageous 
compared  to  other  methods  (Isermann  et  al  [1974],  Saridis  [1974]) 
for  general  linear  processes.  Here,  we  briefly  review  the 
essence  of  the  method.  We  have,  under  the  assumption  of 
stationarity , that  the  autocorrelation  of  the  input  sequence 
{u^}  is 


Ruu(l")  = 


E[uk  Uk-m] 


(3) 


lim 

k-c«> 


1 

k+1 


I 

i=0 


u ■ u ■ 
l i-m 


and  the  cross 


correlation  of 


the  inputs  {u^}  and  outputs 


8 


{zk>  is 


^zu 


= E [zk  uk-m] 


= lim  1 l 
k-*-“  k+1  i=0 


z • u ,•  _ 
1 l-m 


Rzu(m)  = E [zk  uk-m’ 


= E t (yk  + wk)  uk_m] 

— Ryu  + ^wu 

Since  {uk}and  {wk}  are  mutually  uncorrelated. 


Rwu(m)  = °" 


Thus,  R . (m)  = R (m)  (6) 

zu  yu  ' 

The  convolution  equation 

CO 

Ryu*"1)  = l 9(i)  Ruu  (m_i)  (?) 

2 i=0 

relates  Ryu  to  Ruu  through  g(i),  the  impulse  response  of  the 
system.  Thus,  if  an  input  sequence  {ukJ  of  known  autocorrelations 
is  used,  then  estimates  of  g(i)  can  be  computed  from  estimates 
of  RyU ( = Rzu  ) and  Ruu-  In  particular,  if  {uk>  is  chosen 
as  discrete  white  noise,  we  have 


Ruu  ^ = Ruu  • 5i  where  = 1 i = 0 

= 0 i ^ 0 


* 


w^;  that  is  to  say,  we  would  like  to  ensure  improved  perform- 
ance (over  the  scheme  L)  in  terms  of  lower  error  and  lower 
standard  deviations  of  the  error  and  the  estimates  in  the 
cases  where  the  contamination  level  e is,  say,  5-20%  and  the 
contaminating  distribution  C(‘)  has  a large  variance 
and/or  has  heavy  tails  (i.e.  more  outliers).  We  will  not 
mind  if  the  price  to  be  paid  in  adopting  such  a scheme  N is 
that  it  is  worse  - but  not  by  'much'  - than  L in  the  case 
where  e = 0,  i.e.  Wj,  comes  from  a totally  known  distribution. 

We  will  call  the  scheme  N robust  in  the  manner  of  Huber 
[1972]  and  Anscombe  [I960]:  'I  am  willing  to  pay  a premium 

(a  loss  of  efficiency  of,  say,  5 to  10%  at  the  ideal  model) 
to  safeguard  against  ill  effects  caused  by  small  deviations 
from  it;  although  I am  happy  if  the  procedure  performs  well 
also  under  large  deviations,  I do  not  really  care  - inferences 
based  upon  a grossly  wrong  statistical  model  may  have  little 
physical  signif icance . ' 

C.  Proposal  for  a Robust  estimate  of  Rz1a 

We  propose  to  use  a non-linear  scheme  N for  estimating 
the  correlations  Rzu.  It  should  be  mentioned  that  non-linear 
estimates  of  autocorrelations  have  been  proposed  and  used  by 
Huzii  [1963,  1970]  for  stationary  Gaussian  processes  and  by 
Rodemich  [1965]  for  more  general  stationary  processes.  However, 
the  motivation  of  both  these  investigators  was  the  simplifi- 
cation of  the  computational  effort  and  not  the  desensitization 
of  the  estimation  technique  to  different  noise  processes. 


Thus , 


g(m)  = RyU(ni)  = Rzu(m) 

— ^ (8 ) 

Ruu(0)  Ruu ( 0 ) 

The  cross  correlation  estimate,  Rzu(m) , may  be 
computed  recursively  (on-line)  by 

Rzu(m,k)  = Rzu(m,k-1)  +k^1  [zk  uk_m  - Rzu  (m,k-l) ) (9) 

Henceforth,  we  shall  refer  to  (9)  as  scheme  L (for  linear) . 

Using  the  impulse  response  estimates  g(m),  one  can  then  estimate 
the  parameters  a^, . . . ,an,b^, . . . ,b  of  (2)  by  the  method  outlined 
on  p.32  of  Section  1.  Our  principal  concern  in  this  investi- 
gation is  with  regard  to  the  computation  of  the  cross  correla- 
tions  Rzu. 

The  possible  shortcoming  of  scheme  L is  imbedded  in  the 
generality  of  the  noise  w^.  While  it  is  only  required  that 
{wk>  and  {uk}  be  stationary  and  mutually  uncorrelated,  it  is 
natural  to  expect  that  the  performance  of  the  algorithm  L in 
terms  of  rate  of  convergence,  error  of  the  estimation  scheme 
and  standard  deviation  of  the  error,  bias  of  the  estimates 
and  standard  deviations  of  the  estimates  - is  dependent  on 
the  distribution  of  {w^}.  We  propose  to  modify  L (the  scheme 
for  the  on-line  computation  of  Rzu)  for  the  case  where  wk,  in 
addition  to  satisfying  the  usual  assumptions  for  the  correla- 
tions method,  comes  from  a distribution  F (■)  as  specified 
in  the  problem  formulation  (la) . The  goal  is  to  make  the 
computational  algorithm  N less  sensitive  to  the  distribution  of 
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used: 


Consider  again  scheme  L and  the  recursive  algorithm 


zu 


(m,k)  = R (m,k-l)  + 


zu 


A lz * 


uk-m  ~ 


Thus,  as  each  new  observation  zk  is  measured,  the  estimate 
of  R2U (m)  for  (k-1)  measurements,  R2U(m,k-l),  is  updated  by 
adding  [zkuk_m  - R2U (m,k-l) ] . 

We  will  call  the  term  in  parentheses  the  innovation 
Ik  for  the  kth  measurement  zk.  Our  proposal  is  to  use  H (lk) 
instead  of  Ik  in  the  recursive  equation  for  Rzu>  where  H(*) 
is  an  odd,  non-linear  function  as  defined  below  (also  see 
fig.  1).  Thus,  we  have 

Rzu  (m,  k)  = R2u(m,k-1)  + -i-  H[zk  uk_B  - Rzu (m,k-l) I (10) 


H (x)  = 


SjX 


x|  < d1 


= s1d1sgn(x) 
’2 


dliMi  d2 


(ID 


s9d9-s,  d, 

= Sn  (x-(-±-± i_i)sgn  (x)  )d,<  |x|<  (-=-= — ) 

C _ — Q 


= 0 


x > ( 


s2d2”sldl , 


Si  > 0,  s2£0,  d2>d1>0,  d^  < 


H(-)  is  the  estimator  proposed  by  Hampel  [1968,1371] 
to  be  used  in  the  robust  estimation  of  the  location  parameter 
of  a distribution.  It  can  be  seen  that  H(*)  is  the  identity 
function  (i.e.  the  linear  case)  for  si  = s2  = If  d^  = d2  = 00 . 

If  we  choose  S2  = 0 and  d2  = 00 , K is  then  the  well  known 
limiter  function  used  by  Huber  [1964]  for  robust  estimation  and 
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it  has  a constant  value  s^d,  sgn(x)  for  ]x|  d^. 

The  motivation  for  using  a non  linear  H comes  from  an 

examination  of  the  innovations  {i^}.  Although  the  effect  of 

is  damped  at  each  successive  stage  (only  I^/(k+l)  is  added 

to  R (m, k-1) ) , the  characteristics  of  the  noise  in  the  {z,} 
zu  JC 

obviously  influence  the  performance  of  the  algorithm.  We  feel 
that  the  proposed  H,  using  different  values  for  the  parameters 
ST,s2,d-^,d2  - henceforth  called  the  Hampel  parameters  - will 

a 

tend  to  stabilize  the  recursive  computation  of  Rzu  against 
a variety  of  perturbances  {w^} . The  recursive  algorithm  (9) 
used  for  the  linear  scheme  L has  been  claimed  by  Saridis  [1974] , 
without  proof,  to  be  a stochastic  approximation  algorithm 
(Robbins-Mor.ro  [1951]  , Dvoretzky  [1956]  ) , which  would  ensure 
convergence  of  the  scheme  L to  the  true  values  R (m)  and 
give  well-defined  asymptotic  properties  of  the  estimates.  We 
have  not  been  able  to  show,  at  the  present  time,  that  the 
scheme  N of  egn  (10)  using  H ( • ) as  defined  in  (11)  satisfies, 
the  conditions  of  stochastic  approximation  (SA)  (as  in 
Dvoretzky  [1956],  Sacks  [1958]).  On  the  other  hand,  we  are 
optimistic  of  obtaining  these  results  since  the  linear  scheme  L 
satisfies  the  SA  conditions  (Saridis  [1974])  and  the  non-linear 
function  H used  in  scheme  N is  bounded  for  every  non-linear 
estimator  that  we  choose.  This  follows  on  recalling  from  (11) , 
with  the  stated  restrictions  on  the  Hampel  parameters,  that 

H (x)  <_  s^d^sgn  (x)  -V- x 
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To  examine  the  performance  of  different  non-linear 
estimators  for  Rz^  as  compared  to  the  linear  estimator,  we 
have  carried  out  an  extensive  digital  computer  simulation 
for  the  identification  of  three  linear  processes  which  have 
been  used  as  test  cases  by  Isermann  et  al  [1974] . We  have 
used  various  distributions  for  {w  } to  examine,  in  particular, 

iv 

the  effect  of  the  measurement  noise  on  the  identification 
schemes.  The  details  of  the  simulation  and  summarized 
results  are  reported  in  the  ensuing  chapter. 


Chapter  II  Simulation  of  process  identification  using 
different  estimators  for  R 

Z U 

A.  Details  of  Simulation 


The  following  three  simulated  linear  processes 
were  used  (Fig.  2) . 

(i)  Second  order  oscillating  process 

= G^U'1)  - B(Z'1)  = bl2'1  b2z~2 

uk  A(z_1)  l+a1z-1+a2z-2 

where  z~b  -*■  the  unit  delayor 


1 = -1.500 

a2  = 0.700 

1 = 1.000 

b2  = 0.500 

(ii)  Third  order  low  pass  process 

. biz-^-  + b2  z-2  + b3Z“3 

g3(z_1)  - - r ^ — -3 

1 + a^z  1 + a2z  z + a^z 

ax  = -1.500  a2  = 0.705  a3  = -0.100 

b]_  = 0.065  b2  = 0.048  b3  = -0.008 


(iii) 

G22  (z  ) 


Second  order  non  minimum  phase  process 

B (z-1)  _ blz~1  + b2z~2 
A(z-1)  1 + a1z“1  + a2z-2 


-1.425 

a2  = 0.496 

-0.102 

b2  = 0.173 

(12) 


(13) 


(14) 
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Process  (i)  was  used  by  Astrom  and  Bohlin  [ L 9 6 6 ] , Hastings- James 
and  Sage  [1969],  Gustavsson [ 1972 ] , Gentil  [1972]  and  Isermann 
et  al  [1974].  Process  (ii)  has  the  same  parameters  as  the 
third  order  process  used  by  Isermann  et  al  [1974]  , with  the 
difference  that  we  used  no  time  delay.  Process  (iii)  was 
proposed  and  used  by  Isermann  et  al  [1974] . 

The  output  yk  of  the  process  to  be  identified  was 
corrupted  by  w^,  for  which  the  following  20  noise  distributions 
were  used.  Distributions  are  specified  by  means  and  standard 
deviations;  G and  L refer  to  Gaussian  and  Laplacian  distri- 
butions respectively.  We  have  chosen  our  mixture  family  F 
to  be  the  contaminated  Gaussian  class,,  where  the  contaminating 
distribution  C in  synmetric,  since  this  is  the  most  important 
case  from  a practical  viewpoint.  The  contaminations  are 
either  Gaussian  with  larger  variances  or  Laplacian  (double 
exponential)  of  different  variances. 

(Please  see  Table  1 on  next  page.) 
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TABLE  1 


Noise 

u. 

tr 

Nominal  (Known) 
Distribution 

Contaminating 

Distribution 

Contamir 

level 

l 

G (0 , 1) 

L (0 , 1) 

. 05 

2 

.10 

3 

.15 

4 

1.(0, 3. 16) 

. 05 

5 

.10 

6 

.15 

7 

L (0 , 10) 

.05 

8 

.10 

9 

.03 

10 

G (0 , 3 . 16) 

.05 

11 

.10 

12 

.15 

13 

G ( 0 , 10 ) 

. 05 

14 

.10 

15 

.03 

' 16 

G ( 0 , 1) 

— 

“ 

17 

G ( 0 , 3 . 16) 

- 

— 

18 

G ( 0 , 10 ) 

- 

— 

k 19 

G ( 0 , 1) 

L (0 , 3. 16) 

.20 

* 20 

G (0 , 3 . 16) 

.20 

***  Used  as  'basis  for  comparing  estimators  for  different  noises 
* Only  used  for  the  second  order  processes 


These  distributions  are  all  part  of  a library  of  random  functions 
developed  by  the  author  and  A.  K.  El-Sawy  in  the  Electrical 
Engineering  Department  of  The  Johns  Hopkins  University. 

The  input  {uk>  was  discrete  binary  white  noise 
generated  by  using 

v = ran  (0)  -0.5 

(15) 

u = sign  (0.5,  v) 

The  call  ran  (0)  returns  an  uniformly  distributed  number  r, 

O^r <_1 , and  is  reproducible. 
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The  sign  (x,v)  function  returns  |x|*(Sign  of  y) . Thus,  {u^} 
was  a white  binary  sequence  of  amplitude  +0.5  and  -0.5.  So, 
the  autocorrelation  Ruu(0)  = (0.5) 2 = 0.25. 

The  following  estimators  H were  used  in  the  on-line 
computation  of  Rzu(m): 

TABLE  2 


Estimator 


Hampel  Parameters 


# 

type 

S1 

s2 

dl 

d2 

1 

Linear 

1 

1 

OO 

OO 

2 

Limiter 

1 

(Huber  [1964] ) 

1 

0 

2a 

CO 

3 

Limiter 

2 

II 

1 

0 

1.5a 

OO 

4 

Limiter 

3 

II 

1 

0 

a 

OO 

5 

Limiter 

4 

If 

1 

0 

0.7a 

CO 

6 

Hampel 

12A 

(Hampel  [1968] 

1 

-0.27 

1.2a 

3.5a 

7 

Hampel 

25A 

( " ) 

1 

o 

m 

o 

i 

2.5a 

4 . 5a 

Note:  a 

is 

the  standard  deviation 

of  the  known  distribution. 

The  identification  algorithms  for  each  estimator  were 
run  for  3000  stages  of  the  process,  and  parameter  estimates 
were  computed  every  300  stages  using  the  scheme  in  Section  1, 
p.  32.  The  number  of  impulse  response  values,  £,  was  taken 
as  2n.  It  should  be  noted  that  to  improve  the  estimates 
(i.e.  to  reduce  error  and  standard  deviations  of  the  estimates) 
l has  to  be  chosen  very  much  larger  than  2n.  Otherwise, 
because  of  dependence  of  g (£)  for  l > 2n  on  g (1) , . . . ,g (2n) , 
noisy  measurements  increase  the  variance  of  estimates  and 
introduce  large  bias  if  Z is  not  much  larger  than  2n. 
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rp  n dt  f C 


The  error  was  then  computed  as  the  mean  squared 
parameter  errors  related  to  mean  squared  true  parameters: 


Error  = 


11  A 6j  || 

II  ® i ' II 


2n  — 

l (A  6i)2 

i=l 

2n 

I Si2 

i=1  J 


where  (0,f...,8o  ) = (a  . ...,a  ) are  the  true 

-L  Zn  1 n 1 n 

A A A A A A 

parameter  values  and  (81,...,02n)  = (a^«  • • . • • • »bn) 

are  the  parameter  estimates,  and  A0^  = 0^-  0^  . 

To  obtain  better  comparisons  of  the  different 
identification  schemes,  30  Monte  Carlo  runs  of  each  identification 
run  (of  3000  stages)  were  carried  out.  Then  the  means  and 
standard  deviations  of  the  error  (as  defined  above)  and  the 
parameter  estimates  were  computed.  If  j refers  to  each 
Monte  Carlo  run  of  the  identification  algorithm. 


Mean  error  = e = -.r-  £ error  (j) 


std.  dev.  of  error  = sd(e)  = 


r - 2 

2,  (error  (j)  - e) 


mean  parameter  estimates  are 


30 

6i  = 1/30  l 0i  (j) 

j=l 


i=l, . . . , 2n 


M"  ?**•*».  * ^ '•*  « 


r 


std.  dev.  of  parameter  estimates  are 


(18) 


sd(0i)  = 


-,1/2 


30 


1/30  l (e^j)  -ii)2 
j=i  * 


All  the  simulations  were  carried  out  using  the 
facilities  of  the  Westinghouse  Computer  Laboratory  of  the 
Electrical  Engineering  Department  of  The  Johns  Hopkins 
University. 

The  computer  was  a PDP  11/45. 


B.  Summarized  results 


We  have  summarized  the  data  for  all  the  406  examples 
[140  each  for  the  2n<^  order  processes,  126  for  the  3r<^  order 
process]  in  the  following  manner: 

For  each  process,  we  have  chosen  as  our  basis  the 
values  (after  3000  stages)  of 

(i)  the  error 

(ii)  the  standard  deviations  of  the  error 

(iii)  the  std.  deviations  of  the  parameter  estimates 

for  the  LINEAR  estimator  when  wk  is  distributed  according  to 
noise  16,  i.e.  G(0,.l).  We  have  then  taken  the  corresponding 
values  for  all  estimators  for  all  the  noise  distributions, 
and  computed  them  as  percentages  of  the  base  values  [i.e.  the 
linear  estimator  values  for  wv  ^ G(0,1)]  for  that  process. 

If  the  percentage  exceeded  999,  we  took  it,  for  the  purposes 
of  summary,  as  999. 9. Tables  3-7  are  the  summarized  data  for 


and  tables  13-17  are  the  summarized  data  for  process  (iii) . 

To  use  the  tables,  one  needs  to  refer  also  to  tables  1 and  2. 
Any  row  in  the  tables  3-17  has  its  entries  arranged  as  follows: 
The  first  entry  (any  number  from  1 to  20)  specifies 
the  distribution  for  w^  (as  per  table  1) . The  next  entry 
(any  number  from  1 to  7)  indicates  the  particular  estimator 
used  (as  per  table  2) . The  next  two  entries  are,  for  the  just 
specified  noise  distribution  and  estimator,  the  error  and 
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standard  deviation  of  the  error  as  percentages  of  the 
corresponding  values  for  the  linear  estimator  with 

^ G(0,1) (noise  16).  The  remaining  entries  (6  for  the 
3r<2  order  case  and  4 for  the  2n<^  order  cases)  are  the 
percentages  relative  to  the  linear  estimator  for  noise  16  (G(0,1); 
of  the  standard  deviations  of  the  parameter  estimates. 

B. 1 . Summarized  results  for  process  (i) 

The  2a  limiter  has  the  most  uniform  performance 

over  all  the  distributions  for  in  terms  of  the  least  variation 

in  the  6 comparative  values.  For  a given  noise  distribution, 

there  is  not  much  to  choose  between  the  best  estimators  in 

terms  of  bias.  The  2a  limiter  has  lower  values  than  the 

linear  estimator  for  all  but  three  noise  distributions,  and 

is  very  close  to  the  best  performance  for  all  distributions. 

In  most  cases,  it  is  the  best  estimator.  Even  in  the  very 

noisy  case,  i.e.  G(Q,10),  where  almost  all  entries  in  table  7 

(for  noise  18)  exceed  the  upper  limit  (999)  , the  non  linear 

estimates  work  much  better  than  the  linear  estimator.  The 

robustness  of  the  non  linear  estimates  is  well  observed  for 

the  cases  of  high  contamination  (e=0.15,  .20)  and/or  high 

2 

contaminating  variances  (ac  = 10  or  100) . One  can  see  that 
almost  all  the  non  linear  estimators  have  better  performance 
than  the  linear  estimator  in  extreme  cases  like  7 , i.e.  when 
F(w)  = .95  G ( 0 , 1 ) + .05  L { 0 , 10) . 

Next  to  the  2a  limiter,  the  Hampel  25A  estimator 
seems  to  have  the  most  uniform  behavior  over  different  noise 
distributions . 
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It  should  be  noted  that  in  almost  all  cases,  the 
parameter  estimates  have  little  bias.  The  non-linear  schemes 
do  not  significantly  reduce  the  bias  for  the  linear  estimator. 
Finally,  an  important  point  of  note  is  that  most  of  the 
non-linear  estimators  show  marked  improvement  over  the  linear 
estimator  in  the  cases  when  C,  the  contaminating  distribution, 
has  more -outliers.  In  our  simulations,  this  was  achieved 
by  choosing  C as  different  Laplacian  distributions. 

It  is  evident  that,  for  process  (i) , our  proposed 
scheme  N successfully  robustizes,  in  the  sense  defined  before, 
the  on-line  identification  of  process  parameters  by  using 
non-linear  estimators  for  Rzu. 

B.2.  Summarized  results  for  process  (ii) 

This  process,  a third  order  low  pass  system,  was 
found  to  be  difficult  to  identify  and  produced  poor  estimates 
with  any  estimation  scheme,  linear  or  non-linear.  Very  large 
bias  of  estimates,  high  standard  deviations  of  error  and  of 
the  parameter  estimates  were  a common  feature  of  every 
simulation  run.  The  summarized  data  presented  in  table  8-12 
can  easily  lead  to  confusion.  They  show,  for  example,  that 
the  worst  performance  of  the  linear  estimator  is  for  w'v-GCO,!) 
and  the  best  is  for  w^.97  G(0,1)  + .03  L(0,10)!  Thus,  the 
performance  of  all  estimators  for  different  distributions  of 
w are  relative  to  a very  poor  basis  and  this  fact  should 
temper  any  use  of  tables  8-12. 

There  are  only  two  points  we  would  like  to  note  with 
respect  to  the  simulations  for  process  (ii) . Firstly, 

Isermann  et  al  [1974]  also  report  poor  parameter  identification 
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for  process  (ii)  using  correlation  analysis.  Secondly,  an 
examination  of  tables  8-12  (with  the  aforementioned  caution) 
shows,  interestingly  enough,  that  several  of  the  non-linear 
estimators  perform  better  than  the  linear  estimator  for  a 
large  number  of  distributions.  The  Hampel  12A,  the  2a 
limiter  and  the  1.5a  limiter  seem  to  have  the  most  uniform 
performance  over  all  the  distributions. 

B.2.  Summarized  results  for  process  (iii)  ■ 

It  is  difficult  to  determine  which  is  the  most 
robust  estimator  for  this  process.  The  limiter  at  a,  the 
limiter  at  0.7a  and  the  Hampel  12A  - all  show  good  performance 
over  the  different  noise  distributions.  All  of  these 
estimators  give  considerably  better  results  than  the  linear 
estimator  for  all  but  two  of  the  distributions.  The  1.5a 
limiter,  2a  limiter  and  the  Hampel  25A  estimator  also  have 
improved  performance  over  the  linear  estimator,  but  not  as 
uniformly  as  the  first  three.  Of  the  latter  group,  the 
1.5a  limiter  works  best,  followed  by  the  2a  limiter  and  then 
the  Hampel  25A  estimator.  The  non-linear  estimators  for 
this  process  also  reduce  the  bias  of  the  estimates  signifi- 
cantly in  several  instances. 

Finally,  the  improvement  over  the  linear  estimator  is 
again  most  marked  when  the  contaminations  in  w come  from  the 
heavy-tailed  Laplacian  distribution.  As  may  be  expected, 
the  degree  of  improvement  achieved  by  using  the  non-linear 
schemes  increases  with  increasing  level  of  contamination  e. 
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Thus,  for  process  (iii) , our  proposed  scheme  N 
provides  a robust  procedure  for  the  on-line  identification 
of  parameters  through  non-linear  estimation  of  R_„ . The 
additional  benefit  which  arises  for  this  process  from  the 
use  of  these  non-linear  estimators  is  a significant  reduc- 
tion of  the  bias  in  the  parameter  estimates  for  several 
noise  distributions. 

We  have  included,  after  Table  17,  the  computer 
printout  for  5 noise  distributions,  2 each  for  the  second 
order  processes  and  1 for  the  third  order  process.  For  each 
noise,  identification  results  for  all  the  estimators  are  given. 
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Chapter  III  Concluding  Remarks 

It  is  clear  from  the  results  of  the  simulation  that 
our  proposal  provides  a robust  procedure  for  estimating  cross 
correlations  to  be  used  for  parameter  identification.  We 
would  like  here  to  clarify  an  issue  which  is  fundamental  to 
the  problem. 

The  parameter  identification  problem  of  Chapter  I 
can  be  split  into  two  parts.  The  first  part,  which  involves 
the  estimation  of  g (m)  from  estimates  of  Rzu(m),  is  non- 
parametric  and  essentially  infinite-dimensional.  The  second 
part  is  the  transformation  from  this  infinite-dimensional 
characterization  to  the  finite  dimensional  parametric 
representation  of  (2) . We  have  not  addressed  ourselves  at 
all  to  the  problems  associated  with  the  second  part.  This 
was  done  by  following  the  same  procedure  as  in  Isermann  et  al 
[1974]  , viz.  a least  squares  estimation  of  the  parameters 
a^,  . . . ,an,bp  . . . ,bn  which  uses  values  of  the  uncorrupted 

A A 

system  output  y computed  from  g (1) , . . . , g (2n)  by  the  Wiener- 
Hopf  equation. 

Errors  could  easily  arise  from  the  use  of  such  a 
procedure,  especially  since  least  squares  estimation  requires 
uncorrelated  residuals  to  give  unbiased  estimates. 

It  would  thus  be  interesting  to  investigate  the 
following  question:  given  a particular  process  to  be  identified, 

consider  only  the  estimation  of  the  impulse  response  g (m) 

A 

using  our  proposed  scheme  N for  computing  Rzu-  Isermann 
[1974]  reported  that  the  correlation  analysis  produces  the 
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best  estimates  of  impulse  response  in  terms  of  bias  and 
variance  of  the  estimates.  We  feel  it  is  interesting  to 
examine  the  effect  of  our  scheme  N on  the  estimation  of 


g (m) . 


We  mention  in  closing  that  though  our  scheme  N seems 


to  have  worked  well,  an  immediate  question  is  whether  there 
would  have  been  additional  improvement  if  we  had  used,  like 
Isermann  [1976] , 18-22  values  of  the  impulse  response 
instead  of  2n(i.e.  4 or  6).  We  chose  2n  values  for  reasons 
of  computational  cost,  but  hope  to  investigate  the  above 
question  (i.e.,  using  £=18-22)  in  the  near  future. 
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TABLE  3 


SUMMARY 


The  base 
.1151 


1 i 107.3 
105.  0. 
1 G 1 .3 
123*8 
189.  4 
110.1 
106.3 


93*  9 
95.8 

100.1 
116.0 
1 67.  1 
105*9 
95*  1 


96.3 
10  1.8 
111-9 
144.  0 

229-  0 
1 24.  6 
98-4 


124.2 

117-7 

117.9 

141.0 


Or  DATA  FOR  FIRST  SECOND  ORDER  EXAMPLE 


values  are  (for  the  linear  estimator  for  G(0,1)) 


. 0592 

.116 

.158 

.081 

.143 

. j. 

F (w) 

128.4 

106.  0 

108.9 

1 02-  5 

122.4 

• ! 

126.2 

102-6 

1 05.  7 

102.5 

120.3 

136.3 

105.2 

103.2 

103-7 

123..1 

.95 

G ( 0 , 1 ) ■ + .05  L (0,1) 

147.3 

117.2 

118.4 

119.8 

152*4 

I 

274.2 

200.0 

183.  5 

156.8 

249.  0 

133.9 

115*5 

105.7 

1 09.  9 

1 32.  9 

-- 

1 - 0 

128-9 

105.2 

107.  O' 

1 03.  7 

122.4 

• ;;  - 

91-2 

87.  1 

91. 1 

91.4 

102.8 

* * " K 

91.9 

90.5 

91.1 

96.  3 

104.2 

8-1.8 

91.4 

91-8 

■ 102.5 

105.  6 

.9 

g ( o , l ) + .1  L(a,u  . 

93.2 

1 09.5 

107.0 

112.  3 

119.6 

- 

152-2 

161.2 

152.  5 

144.4 

18  6.  0 

8 3*6 

90.5 

100.6 

1 07.4 

109.  8 

*. 

93-  1 

87.  1 

93.  0 

92.  6 

103.5 

149.  8 

105.2 

110.8 

91.4 

118.9 

1 38.  9 

110-3 

107.  6 

93.8 

119.  6 

125.8 

1 12.  1 

112.0 

101*2 

125.  9 

in 

00 

G(0,1)  + .15  L ( 0 , 1 ) 

159.8 

141.4 

'149.4 

123.5 

157.3 

266.4 

238.6 

234*2 

1 72-8 

251-7 

131.8 

1-32.8 

120-3 

1 12-  3 

134.  3 

147-0 

110.3 

108*9 

91.4 

118-2 

1 63.5 

134.5 

127.8 

103-7 

146-9 

147.0 

119.0 

121-5 

102-5 

137.  1 

151-0 

126.  7 

119.0 

103.7 

139*9 

164-0 

141.4 

134.2 

118.5 

-95  G ( 0 , 1 ) + .05  L (0 , 3. 16) 

319.  6 

227.6 

213.  3 

160*5 

28  1*8 

1 53.4 

126-7 

126.  6 

109.  9 
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124.  1 
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137.  1 
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i J:d 

5 
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101.7  . 

105.2 

1 14.  6 

109*9 

113-3 

5 

4 

138.6 

139.  0 

1 35*  3 

143.7 

124.  7 

142.  0 

.9 

G(0,1) 

5 

5 

. 196.4 

231.6 

201.7 

200.6 

159.3  - 

226.  6 

5 

6 

119.7 

1 19-4 

117.2 

124.  1 

U17.3 

119.6 

5 

7 

C 

103.5 

104.  1 

100.0 

108-2 

96*3  . 

1 04.  2 

i 

1 

6 

1 

155.3 

231-3 

• \ 

175.9 

170.9 

98.8 

189.  5 

> 

6 

2 

- 131.0 

163.9 

137i9 

• 140.5 

97.  5 

147.6 

6 

3 

140.3 

147.  1 

149.1 

140. 5 

103.7 

151.7 

. 85 

0(0,1 

6 

4' 

180.1 

208.  1 

133.6 

'■191.3 

127-2 

20  1.4 

6 

5 

2S3.7 

415*5 

339.7 

296.8 

174.  1 

355.2 

6 

6 

156.0 

146.5 

154.3 

158.2 

113.  6 

164.3 

6 

7 

130*8 

169-4 

141.4 

140.5 

97.  5 

148.3 

7 

1 

257.9 

342-  1 

277.  6 

. 284.8 

137.0 
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7 
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172.  1 

139.7 
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102.  5 

155.  9 

7 
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162-7 

133-6 
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151.7 
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r*  ! r\  i 
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124.7 

180.4 
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7 
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144.9 
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8 
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145.4 
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F (w) 

+ .1  L (0 , 3 . 16) 


+ .15  E ( 0 , 3 . 16 ) 


+ .05  L ( 0 10 ) 


+ .1  L ( 0 , 10 ) 
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297-8 
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241. 1 

119.8 

236.4 

F(w) 

9 
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9 
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14 
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14 
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1 
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15 
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3 - 

4 
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191.2 
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142.  0 
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15 

5 
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& 
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133.1 
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127.2 

145.5 

/ 

15 

7 
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154.7 
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101.2 

124.  5 
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100.0 
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16 

2 

97.7 

103.4 
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101.2 

95.8 

16 

3 
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107.4 
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16 

4 
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141.4 

146.  8 
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169.2 

16 

5 
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228.4 

241.8 

1 69.  1 
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G ( 0 , 1 ) 

1 6 

6 
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138.7 
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114.8 

137.8 
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99.2 
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303.  A 

38  0 . 2 

307.8 

325.3 

ISA.  0 

331.5 

17 

2 

297.  0 

351*5 

318.  1 

306.3 

182.  7 

31  A.  7 

17 

3 

293*7 

3A3.  2 

317.2 

30  1.3 

181 . 5 

307.7 

1 7 

290.  A 

3 1 A.  5 

2S6.2 
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1 7 

5 
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377.  A 
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17 

6 
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1 7 

7 
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13 

1 
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18 

2 
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18 
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A 
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5 

999.  9 

999.9 
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18 

6 
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18 
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TABLE  8 


SUMMARY  OF  DATA  FOE  TRIED  ORDER  EXAMPLE 
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the  linear  estimator 
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F(w) 
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3.53 
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14.5 
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9.9 

13-4  . 
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■’  15-6 

12-3 

12.3 
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- 
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o 
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12.  5 
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12-0 
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13.7 
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1 

4 
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42-0 

46*4  * 
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4 

6 

99.3 

104.  0 

97-2 

124.  8 

101-6 

80.6 

4 

7 

101.7 

111-0 

106-4 

124-  1 

192-  1 

95.9 

ui  x>  uro  ►-  , -i  O'  cn 


\ 

TABLE  14 


5 1 
5 2 
5 3 
5 4 


135.6 

144.4 

162*  0 

154.0 

112.7 

143-9 

, * 

10  0-3 

102.6 

81.7 

135.4 

133.2 

110-2 

97.4 

108.  6 

93.0 

127.  5 

100. Q 

. 92.9  “ . . ■ 

96-7 

94. 3~ 

105*  0 ( 

1 0 9.  9 

* . 

“ 85.  7 

89"3  .9  G(0,l)  + vi 

L (0,3. 16) 

113.5 

1 75.8 

168.4 

143.7 

103.2 

125.5 

99.8 

121.8'“ 

103-7 

133-9 

111.1 

- 74.  5 • •. 

* 7.  - ‘ •* 

100-1 

.81.4 

94-7“ 

112-5 

100.0- 

10  6.1.  /»;-* 

* , '•  - * 

190.6 

235.9 

248*  6 

219.7 

83.9 

‘ , ..  ' V i*  * . 

167.3  ; 1 - ' 

138.3 

160.8 

171-2 

161.7 

123-8 

151-0 

117.0 

1 06.3 

122-2 

135.  6 

•79-4 

99-0  , / 

122*  1 

116.1 

137.2- 

: 132.4 

82-5 

105.1 

4 

124.4 

113-6 

135-2 

137.4 

82-5 

118.4  - 

• • - - . ~ ' ' - . 

120.3 

139.2 , 

0.  0 

999.9 

999.9 

45.  9.85  G (0, 1)  +. 

15  L(0 , 3.,16)  • 

133.9 

1.54.3 

170.6 

145.0 

141.3 

173-5  “ 

225.2 

362. S 

345.7 

108.6 

114.3 

103.1 

10  1.6 

93.  0 

104.4 

100.1 

111.6 

111.4 

96*  1 

129.1 

“ 129-  5 

96*5 

37.7 

31.1 

110.7 

10  3.4 

104.  O' 

282*  8 

230*2 

519-4 

134.  6 

I 03.2 

101 .0 

1 13.  0 

127.0 

90.8 

123.  1 

1 65.  1 

122.4 

1 17-5 

1 09.5 

84-7 

116.Q 

122.2 

75.  5 

129.7 

95.2 

112.2 

95  G ( 0 , 1 ) + . 0 5 L ( 0 , 10 ) 


s' 

1 

252.2 

285.  5 

256*  1 

285.  2 

184.  1 

8 

2 

126.  6 

168.5 

131*7 

166-3 

100-0 

S 

3 

139.4 

277.3 

213-4 

221*2 

157-  1 

3 

4 

i *73  • 2 

728.8 

710.8 

109-8 

1 04.8 

S 

5 

177.9 

652-8 

d03*  3 

403.2 

95.2 

3 

6 

1 1 0 • 2 

119.  1 

120-9 

127-  5 

79.4 

8 

7 

149*5 

210.7 

183-4 

186.  7 

90.5 

122 


225.5 

137-8 

193.9 

365.3 

259.8  9 G(0,l)+.i  L (0 , 10 ) 

1 0 7 • 1 

14U8 


TABLE  15 


9 

1 

2 0 0 » 3 

240.3 

256.3 

203.7 

£32*  5 

357.  i 

9 

2 

100.2 

71*3 

101.3 

10  6.7 

115*9 

104.  1 

9 

3 

65*  £ 

67.  7 

8S.6- 

94.  1 

95.  2 

101*0 

9 

4 

65»  S 

59»  1 

69.  5 

£7  • 0 

95.  2 

87-  o 

9 

5 

90.7 

53.9 

91 . 6 

94.  1 

87.3 

83.8 

9 

6 

S3  » 3 

60.2 

91.7 

90.9 

90*5 

1 04.  1 

9 

7 

. 102.9 

_67 . 2 

99.4 

1 i 0. 3 

77.6 

111-2 

F (w) 


.97  G (.0 , 1 ) + . 0 3 L (0 , 10) 


1 0 

1 

1 0 o » 2 

164.7 

90.6 

175.4 

1 0 

2 

98  » 8 

139.  9 

87.5 

149.2 

10 

3 

99-2 

139*6 

90*8 

150.9 

10 

4 

125-4 

322.5 

150*6 

292.4 

1 0 

5 

92-0 

97.  9 

81.3 

115.2 

10 

6 

100-4 

125.  6 

97.8 

140.0 

1 0 

7 

98-  1 

122-  1 

82.4 

13  6.  6 

136.5  96. 0 

134.9  106.1 

101.6101.0 

16^.1  I6A.3.95  G ( O',  1 ) + . 05  G(0,3.16) 

147.6  96.9 

126.6  101-0 

1 55.  6 10  3-  1 . 


1 1 

1 

104.3 

1 0 9*2 

91 .0 

126-3 

1 1 

2 

122-2 

235*2 

104.4 

235.4 

1 1 

3 

108.9 

194.1 

92.  1 

1 98*  5 

11 

4 

143.5 

392-9 

175.2 

355.8 

1 1 

5 

98.3 

90.3 

88.3 

114.7 

1 1 

6 

99.3 

108.  0 

94.1 

129.3 

1 1 

7 

94.5 

62.4 

87.2 

9 6-5 

114.3  124*5  , 

133-3  132.7 

61.0  123.5 

130.2  134-7  .9  G ( 0 , 1')  + . 1*G  (0 , 3 .16) 

141.3  91.8  . , 

134. 9 104.1 

215.9  115-3 


12 

1 

1S5.  6 

431-4 

181.7 

447-  6 

157.  1 

155*1 

12 

2 

161-7 

581-8 

208.3 

517.  9 

206.3 

178.6 

12 

3 

122.9 

160.5 

117.1 

1 65.  6 

274.  6 

152.0 

12 

4 

ICS. 4 

118.9 

107.3 

1 32.4 

1 68-3 

120.4 

12 

5 

115-4 

131.4 

101.3 

150.8 

1 66.  7 

108.  2 

12 

6 

429.7 

999.  9 

999.9 

483.  0 

999.  9 

207.  1 

12 

7 

17  6*7 

434.8 

151.4 

205-2 

999-9 

999-9 

85  G ( 0 , 1) + . 15  G ( 0 , 3 . 16) 


TABLE  16 


13 

1 

192.7 

282-4 

187.7 

282.  6 

13 

2 

103.4 

130.0 

118-2 

139-3 

13 

3 

95.0 

135-  6 

86-8 

143.0 

13 

4 

11  6.2 

256*  5 

113-8 

246-  0 

13 

5 

101-  1 

117-7 

36.6 

136.8 

13 

6 

92-3 

98.5 

83-3 

1 1 6*  4 

13 

7 

10  1.2 

98.5 

95-2 

123.  1 

14 

1 

269-3 

■505-6 

288.3 

450.3 

14 

2 

114.0 

128.6 

111.9 

124.  1 

14 

3 

10  1.8 

89-  0 

111.6 

100.5 

14 

4 

1 17-5 

193-3 

99.4 

204.4 

14 

5 

124.7 

157.8 

135.4 

157-2 

14 

6 

110.0  . 

110-0 

104.0 

137.3 

14 

7 

147.3 

238.5 

151.7 

241. 1 

15 

1 

130-1 

127.  1 

120.2 

147. 

6 

1 5 

2 

117.5 

200.2 

193.0 

138- 

6 

15 

3 

96-4 

106.6 

107.3 

1 16- 

/. 
— * 

15 

4 

95-9 

127-2 

9S  » 0 

135- 

9 

1 5 

5 

95-2 

118-2 

92.  1 

130. 

4 

15 

6 

10  7.2 

140-4 

133.8 

140- 

8 

15 

7 

110-0 

198-4 

128.6 

192- 

1 

223.6 

141.8 

F (w) 

177-8 

123.  6 

182.5 

105.1 

76.2 

117.3 

-.95  G(0,l)+.05  G ( 0 , 10 ) 

117.5 

96.9 

150.8 

110.2 

144.4- 

120.4 

282-5 

231.6 

207.9 

122.4 

257-  1 

135.  7 

- - 

127.  0 

104.  1 

147.  6 

98.  0 

139.  7 

85.  7 

96.3 

108.2 

.9  G(0,l)+.l  G ( 0 , 10 ) 

195*2  120*4 

193.4  111.2 

168.3  123.fr 

176.2  120.4 

114*3  98*0  G(0,l)+.03  G(0,10) 

150*8  134*7 

82 * 5 129.6 


1 6 

1 

10  0-0 

100-0 

10  0.0 

100.0 

100.0 

100-0 

1 6 

2 

98-2 

96-  9 

97.  1 

94.  3 

10  7.9 

98-0 

1 6 

-j 

1 G 1 • 5 

121.2 

120.2 

99*  7 

93-7 

109.2 

13 

4 

110.0 

158.5 

1 65*  5 

1 0 6.  4 

12  2-7 

136.  7 

1 6 

5 

10  6.1 

147-  5 

149.5 

99.  0 

111.1 

121.4 

16 

6 

10  0.9 

117.5 

114.5 

10  3.0 

84.  1 

116-3 

1 6 

7 

100-0 

100.0 

10  0.0 

100.0 

100-0 

100-0 

G (0 , 1) 
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TABLE  17 


17 

1 

213-2 

189.8 

169.7 

23  1*2 

263-5 

179-6  F(w) 

1 7 

2 

213*0 

190-0 

1 69.  9 

231. 0 

261 . 9 

1 78  • 6 

17 

n 

273*  i 

5 59.9 

244.2 

541-0 

271.4 

204.  1 

17 

^4 

439*  7 

999-9 

498.  0 

999.9 

560-3 

331-6 

1 7 

5 

247  • 1 

394.8 

172-7 

4U  0. 2 

25  5*6 

189.8  G(0,  3.16) 

17 

6 

249-  5 

334.2 

243-9 

346.  5 

250-8 

249.  C 

1 7 

7 

213-3 

189-8 

1 69. 7 

231-2 

263-5 

179-6 

18 

1 

290.4 

271.3 

185.0 

350. 1 

0(1*4 

595-9 

18 

2 

290-5 

27 1-3 

135.0 

350.3 

671*4 

595.  9 

18 

3 

294.  0 

304.8 

132.3 

377.4 

671.4 

591-8 

IS 

4 

253.6 

241.3 

207.2 

307*6 

677-3 

591-8 

18 

5 

252*6 

206.  1 

162-0  . 

252-  6 

701-6 

£!:?  g ( o , io ) 

18 

6 

32  9-3 

386.2 

255.8 

414.3 

679.  4 

IS 

7 

290.4 

271.3 

135.0 

350.  1 

671-4 

595.  9 

19 

1 

157-9 

167-3. 

200.0 

1 56-  5 

127.  0 

239.8 

19 

2 

140-3 

1 95-  6 

230-3 

126.  6 

95.2 

200.0 

19 

3 

131-0 

227.3 

248-  1 

1 16.4 

93.  7 

203-  1 

19 

4 

105-0 

96-4 

133-8 

10  1.0 

109.5 

155-1 

19 

5 

97-  1 

77-7 

116.0 

95-4 

100.0 

]63*3  ‘8  G(0'1)+-2  K0, 3..  16) 

19 

6 

134.8 

249-3 

2S2-8 

2 0 6.  4 

101-6 

19 

7 

140-3 

161-9 

194.9 

135-  2 

111.1 

157-8 

20 

1 

135*6 

135.9 

K> 

to 

9 

146.4 

181-0 

155.1 

20 

2 

146»6 

188.4 

164.2 

191.6 

174-6 

128. 6 

20 

3 

1 79*  5 

372.  6 

350-  1 

243.  5 

347.  6 

130-6 

20 

4 

138.  1 

213-6 

130.5 

220.4 

146-  0 

117.3 

20 

5 

1 29-  9 

1 52.  5 

116*7 

172  - 8 

192.  1 

117-3.8  G ( 0 , 1 ) +.  2 G ( 0 , 3 . 16 ) 

20 

6 

141-2 

211-0 

125-9 

2 28  * 0 

150.8  ■ 

121.4 

20 

7 

161-1 

257.0 

149.9 

253.  3 

263-5 

146-9 

125 

I 

9 


BREEDING  PAG: 

V *wr  : 


BLANK—  iOT  FILMS 


' <* 


COMPUTER  PRINTOUTS 


127 


System  order  n = 2 


The  system  parameters  are(in  the  ordercl,  . . i an-bl , . . < bn)  : 
-1.500 
0 700 
0 GOO 
1.000 
0 5 00 
0.000 


Nominal  noise:  Gaussian 

Contaminating  noise:  Laplacian 
Contamination  level  * 0 10 

The  Hampel  parameters  are: 


Mean=  0.00  Std  dev*  1.00 

Mean*  0.00  Std.  dev*  3.16 


si  * 10  0 s 2 ■ 1.00 

dl  * 0.17*  38  d2  = 0 . 1 7 e 38 

ucutof  * 0 . 1 7e  38 


Stages  run 
Std  . 


300  £r  ror  = 
dev.  of  error* 


0 .4467 
0 .2669 


Parameter 
est i mates 
-1.572 
0 .7  66 
0 964 
0 .453 


Std.  dev.s  of 
est i nates 
0.485 
0.671 
0.213 
0 . 580 


Stages  run  600  Error  * 
Std.  dev.  of  error* 


0 .3826 
0 .3025 


Parameter 
estimates 
-1  543 
0.706 
1007 
0.490 


Std.  dev.s  of 
est i nates 
0.452 
0 616 
0.197 
0 . 570 


Stages  run  900 
Std.  dev 


Error  = 
of  error* 


0 .2640 
0.1781 


Parameter 
estimates 
- 1 .4  78 
0 664 
0 999 
0 554 


Std  dews  of 
estimates 
0 . 280 
0 411 
0 174 
0 350 


Stages  run  1200  Error 


0.2156 
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Std 

dew.  of  error® 

0.1476 

Parcneter 
e st i nates 
-1  4 97 

0 689 

1 .004 
0 .5  26 

Std  dews  of 
e st  i notes 
0 . 233 
0 339 
0 . 126 
0 294 

Stages  run 
Std  . 

1500  Error  * 

dew . of  error® 

0.2106 

0.1540 

Parai'.eter 
e st i nates 
- 1 5 19 
0.704 
0 996 
0 497 

Std  dews  of 
e st  i na  te  s 
0 232 
0 . 332 
0 110 
0 309 

Stages  run 
Std  . 

1800  Error  ® 

dew . of  error® 

0 . 1 935 
0.1781 

Parcneter 
e s t i nates 
-1.536 
0 726 
0 993 
0.477 

Std  dews  of 
e st  i na  te  s 
0 239 
0 . 323 
0.101 
0 319 

Stages  run 

2100 

Error  ® 

0.1818 

Std  . 

dew. 

of  error® 

0 . 1 633 

P ar  a ne  te  r 

Std 

dev  s of 

e st i na  te  s 
-1  519 
0 705 
0 999 
0.496 

e st  i na  te  s 

0 214 
0 313 
0 086 
0 . 294 

Stages  run 

24  00 

Error  « 

0 . 1594 

Std 

dew. 

of  error® 

0.1072 

Parameter 

Std 

dew . s of 

est i nates 
- 1 521 

0 702 

1 009 
0.479 

e st i na  te  s 

0.181 
0 . 235 
0 087 
0 226 

Stages  run 

2700 

Er  ror  ■ 

0 1 446 

Std 

dew 

of  error® 

0 1034 
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Parameter 
nt  i nates 
-1.539 
0.719 
1 .004 
0 .464 


Std  dev  . s of 
•st i nates 
0 . 155 
0.221 
0 . 082 
0 . 208 


Stages  run  3000  Error  * 0.1342 

Std.  dev.  of  error*  0.0712 


Parameter  Std.  dev.s  of 
estimates  estimates 

-1.526  0.138 

0 .703  0.185 

1.003  0.078 

0.484  0.178 
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System  order  n * 2 


The  system  parameters  are<in  the  orderaii  . . . an ib 1 / . . < bn)  s 
- 1 .500 
0.700 
0 0 00 
1.000 
0.500 
0 0 00 


Nominal  noise:  Gaussian 

Contaminating  noise:  Laplac«an 
Contamination  level  * 0.10 

The  Hampel  parameters  are: 


Mean*  0.00  Std.  dev  = 1.00 

Mean®  0.00  Std.  dev  .*  3.16 


si  3 1.00  s 2 * 0 00 

d 1 3 0 . 2 Oe  01  d 2 * 0 l?e  38 

ucutof  = 0.1?e  38 


Stages  run 
Std  . 


300  Error  ■ 
dev.  of  error= 


0.4106 
0 .3293 


Parameter 

Std.  d ev  . s of 

est i mates 

est i mates 

-1.581 

0 . 486 

0 777 

0 . 622 

0 947 

0.221 

0.427 

0 . 642 

Stages  run 

600 

Error  * 

Std  . 

dev. 

of  error* 

Parameter 

Std 

dev  s of 

est i mates 

est i nates 

- 1 582 

0.518 

0 773 

0 . 655 

0.998 

0 . 198 

0 424 

0 70  1 

Stages  run 

900 

Error  = 

Std. 

dev. 

of  error* 

Parameter 

Std 

d ev  s of 

estimates 

estimates 

- 1 .534 

0 310 

0 7 20 

0.419 

0 990 

0.173 

0 4 97 

0 399 

0 .3864 
0 .4038 


0 271  1 
0 .2059 


Stages  run  1200  Error  * 0.2177 
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Std.  dev.  of  irror*  0.1475 

Parameter 
• st  mates 
-1.514 
0.711 
0 995 
0.512 


Stag*s  run  1500  Error  * 0.1993 

Std.  d*v.  of  error*  0.1118 

Paraeeter  Std.  d*v.s  of 
estinates  *stinat*s 

-1.526  0.211 

0.701  0.289 

0 .988  0.117 

0.506  0.257 


Stages  run  1800  Error  * 0.1699 

Std.  d*v.  of  error*  0.1251 

Par aneter 
•st i nates 
-1 .529 
0.712 
0 .987 
0 .494 


Stages  run  2100  Error  * 0.1655 

Std.  dev.  of  error*  0.1215 


Std.  dev . s of 
est i nates 
0.191 
0 262 
0 . 106 
0 . 245 


Std . dev.s  of 
est i nates 
0.231 
0 . 343 
0.131 
0 . 295 


Paraneter  Std.  dev.s  of 
estinates  estmates 

-1.507  0.175 

0 703  0.269 

0.992  0.093 

0.510  0.237 


Stages  run  2400  Error  * 0.1488 

Std.  dev.  of  error*  0.0909 

Paraneter  Std  dev.s  of 
estinates  estinates 

-1511  0.154 

0 699  0.223 

1.001  0.091 

0.493  0.198 


Stages  run  2700  Error  ■ 0.1348 

» j Std.  dev.  of  error*  0.0784 

I 
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Paroneter  Std  dev  .s  of 


est i nates 

est i notes 

- 1 528 

0.133 

0 7 13 

0 202 

0.999 

0.084 

0.479 

0 . 174 

Stages  run 

3000 

E r r 0 r «= 

Std  . 

dev. 

of  error= 

Paroneter 

Std 

dev .s  of 

est i notes 

est  i notes 

-1  519 

0 118 

0.705 

0 173 

1 .002 

0.081 

0.488 

0 153 

0 .1228 
0 .0595 
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Systen  ord»p  n * 2 


The  systen  paran*t»rs  ar*(  in  the  order  a 1 > . . » an < b l , . . , bn  ) : 


-1.500 

0 700 
0 .000 

1 000 
0 .300 
0 .000 

Nonina!  noise:  Gaussian 

C on t an i n a t i ng  noise:  Lapl&cian 
C on t an i na t i on  level  * 0.10 

The  Hanpel  paraneters  are: 


Kean*  0.00  Std.  dev*  1 00 

Mean*  0.00  Std.  dev.*  3 16 


si  ■ 1.00  s 2 • 0 .00 

dl  * 0 . 1 3*  01  d2  * 0 1 7 e 38 

ucutof  » 0 . 1 7e  38 


Stages  run 
Std . 


300  Error  » 
dev.  of  error* 


0 .3863 
0 .3730 


Paraneter 
est i na  te  s 
-1.361 
0 .733 
0 .930 
0 .432 


Std  dev . s of 
est i nates 
0 . 348 
0 . 388 
0 . 206 
0.671 


Stages  run  600  Error  * 
Std.  dev.  of  error* 


0 .3909 
0 .4879 


Paraneter 
est i nates 
-1.587 
0 .773 
0.983 
0.414 


Std . dev .s  of 
est i nates 
0 . 607 
0 . 689 
0 . 193 
0.812 


Stages  run 
Std . 


900  Error  * 
dev.  of  error* 


0.2811 

0.2415 


Paraneter 
e st  i na  te  s 
-1.334 
0 . 727 
0.983 
0 .488 


Std.  dev.s  of 
est i nates 
0 . 347 
0 . 438 
0 . 173 
0 450 


Stages  run  1200  Error  * 0.2349 

135 


. v. 


J ur.  b 0 1:02 

noise!  Page  2 

Std  . 

dev.  of  error=  0.1841 

Parameter 

Std.  dev.s  of 

est j nates 

est inates 

- 1 5 18 

0 283 

0 719 

0 . 382 

0 989 

0.138 

0 502 

0.347 

Stages  run 
Std  . 

1500 

dev. 

Error  = 

of  error* 

0.2117 

0.1137 

Parameter 
est i nates 
-1.518 
0.896 
0.982 
0 5 12 

Std.  dev.s  of 
est  i na te s 
0.215 
0.301 
0 . 124 
0 . 279 

Stages  run 
Std 

1800 

dev. 

Error  = 

of  error* 

0 . 1788 
0.1100 

Parameter 
est i nates 
-1.509 
0.710 
0 .982 
0 . 504 

Std.  dev.s  of 
estimates 
0.186 
0.260 
0.113 
0 . 245 

Stages  run 

2100 

Error  = 

0.1698 

Std  . 

dev. 

of  error* 

0.1087 

Parameter 

Std 

dev.s  of 

est i nates 

estimates 

-1.503 

0.161 

0.688 

0 . 255 

0 .986 

0.101 

0.521 

0 . 230 

Stages  run 

2400 

Error  = 

0 . 1532 

Std  . 

dev. 

of  error* 

0 .0936 

Parameter 

Std 

dev.s  of 

estimates 
-1.505 
0.692 
0 995 
0 506 

estimates 
0.164 
0 . 223 
0 099 
0 . 205 

Stages  run 

2700 

E r r o r * 

0.1408 

Std 

dev. 

cf  error* 

0 .0767 

136 


'■•it;  * • 


J 


Jun  6 01:02  noiseS  Pag*  3 


Paran*t«r  Std.  d*v.s  of 
estinates  estinates 

-1 .518  0.140 

0.703  0.202 

0.994  0.092 

0.491  0.182 


Stag*s  run  3000  Error  » 0.1302 

Std.  dev.  of  »rror»  0.0602 

Paran»t*r  Std.  d*v.»  of 
•st i nates  estinates 

-1  .316  0 . 122 

0 690  0181 

0 997  0.089 

0 499  0.162 


J u n 6 01:12  n o ; s e 5 Page  1 


System  order  n * 2 


The  system  parameters  ar«(  in  the  order  a 1 < . . < an  . b 1 « 
-1.500 

0 .700 
0 0 00 

1 0 00 
0.500 
0.000 


bn) 


Hohinal  noise:  Gaussian 

Contaminating  noise:  Lap  Uc  i an 
C on t am i n a t i on  level  = 0.10 

The  Hampel  parameters  are: 


PI  ea  n*  0.00 
II  e a n = 0.00 


si  * 1.00 
dl  = 0 . 1 Oe  01 


Std  . dev  . = 
St  d . dev.» 


s 2 = 0 .00 
d 2 = 0 . 17e 


uc  u to  f = 0 . 1 7 e 38 


Stages  run 

300 

Error  = 

0 4494 

Std. 

dev. 

of  error* 

0.5282 

Parameter 

Std 

d ev . s of 

est i nates 

estimates 

- 1 5 68 

0 .'654 

0 .788 

0.804 

0 9 01 

0.204 

0.407 

0 879 

Stages  run 

600 

E r ro  r = 

0.4169 

Std  . 

dev. 

of  error* 

0.5837 

Paraneter 

Std 

dev . s of 

est i nates 

e 

st  i na te s 

- 1 610 

0 . 788 

0 .780 

0 . 724 

0 .951 

0.181 

0 .392 

0.918 

Stages  run 

900 

Error  = 

0.333  1 

Std 

dev. 

of  error* 

0.3521 

Paraneter 

Std 

dev .s  of 

est i mates 

est i mates 

-1  553 

0 436 

0 .758 

0 587 

0 955 

0 16  9 

0 456 

0 603 

Stages  run 

1200 

E r r o r = 

0 27*7 

■ r.  H 

. , *«•. 

« evt*'  • • 

138 


1 . 00 
3.16 

38 
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Std.  dev.  of  error=  0.2423 


P ar  a ne  te  r 

Std 

d ev  . s 

of 

e st i nates 

est i nates 

-1  .534 

0 . 354 

0 .733 

0 . 447 

0 .966 

0 143 

0 .486 

0 . 432 

Stages  run 

1300 

Error 

3 

Std 

de  v . 

of  error® 

Paraneter 

Std 

. dev . s 

of 

e st i na  te  s 

est i nates 

-1.312 

0 . 265 

0.713 

0 . 366 

0 .961 

0 . 132 

0.503 

0 . 346 

Stages  run 

1800 

Er  ror 

3 

Std 

de  v . 

of  error® 

Par ane  ter 

Std 

dev  . s 

of 

e st i na  te  s 

e st  i na  te  s 

-1  307 

0 . 223 

0 .709 

0.314 

0 961 

0.119 

0 .308 

0 . 283 

Stages  run 

2100 

Er  ror 

3 

Std . 

de  v . 

of  error® 

Paraneter 

Std 

d ev  . s 

of 

e st i na  te  s 

est i nates 

-1.507 

0.216 

0 .683 

0 . 295 

0 .967 

0.111 

0 .324 

0 . 264 

Stages  run 

2400 

E r ror 

3 

Std . 

de  v . 

of  error® 

Par aneter 

Std 

dev  .s 

of 

e st i na  te  s 

est i nates 

-1.302 

0 . 184 

0 694 

0 . 276 

0 973 

0.109 

0 309 

0 233 

Stages  run 

2700 

E r ror 

3 

Std . 

dev. 

of  error® 

0 .2301 
0.1332 


0 .2164 
0.1213 


0 .2008 
0.1191 


0.1839 
0 1111 


0 168  1 
0.1006 


139 
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Parameter 

Std  d ev  . s of 

est  i nates 

est i nates 

-1.508 

0.168 

0 703 

0 . 250 

0 976 

0 . 105 

0 .499 

0.226 

Stages  run 

3000 

Error  = 

0.1595 

Std  . 

dev. 

of  error1 

0 .0823 

Parameter 

Std 

dev .s  of 

est i nates 

est i nates 

-1.511 

0.157 

0 686 

0.227 

0 979 

0.101 

0.509 

0.203 
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System  order  n 3 2 


The  system  parameters  are(  in  the  orderoli  . . . m - bl  i . . / bn  ) i 
-1.300 
0.700 
0 .000 
1 .000 
0 .500 
0 .000 

K 

Nominal  noise:  Gaussian 

Contaminating  noise:  laplacian 
C on t am i n a t i on  level  3 0 10 

The  Hampel  parameters  are: 


H«a n*  0.00 
Neon*  0.00 


St  d dev.3  1.00 
Std.  dev.3  316 


si  3 1.00  s 2 = 0 .00 

dl  3 0 . 7 Oe  00  d 2 3 0 . 1 7 e 38 

ucutof  3 0 . 1 7 e 38 


Stages  run  300  Error  3 
Std.  dev.  of  error3 


0.6014 
0 331  9 


r 


P ar  a me  te  r 

Std . dev .s  of 

est i mates 

estimates 

-1 .653 

0 . 925 

0 .735 

0 . 806 

0 869 

0 . 238 

0 .303 

1.001 

Stages  run 

600 

Er ror  3 

Std. 

de  v . 

of  error3 

Parameter 

Std 

d ev  . s of 

estimates 

estimates 

-2.210 

3 . 986 

2.419 

9 308 

0 .953 

0 . 347 

-1.169 

8 . 885 

Stages  run 

900 

Error  3 

Std . 

dev. 

of  error3 

Parameter 

Std 

dev . s of 

estimates 

estimates 

-1.397 

0 . 663 

0 817 

0 . 856 

0 .926 

0 20  1 

0 .332 

0.913 

1 .6410 
6 .6624 


0.4481 
0 .3706 


Stages  run  1200  Error  3 0.3700 

141 


r *?  • . 14  - 
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ist 5 Page  2 

Std.  dev 

of  error5 

0 .3590 

Para ne ter  Std 

d ev  s of 

estimates  est 

i nates 

_ « e e -* 

“1  j 3 i 

0 

to 

CO 

0 - 762 

0 

610 

0 936 

0 

. 175 

0.414 

0 

. 637 

Stages  run  1500 

c 

r ro  r = 

0 3269 

Std.  dev. 

of  error5 

0.2212 

Paraneter  Std 

dev .s  of 

estimates  est 

i nates 

- 1 .528 

0 

370 

0 .725 

0 

459 

0.934 

0 

.162 

0.452 

0 

. 490 

Stages  run  1800 

c 

r r o r = 

0 .294  1 

Std.  dev. 

of  error5 

0 1 962 

Para  n e ter  Std 

dev.  of 

estimates  e 

St 

i nates 

- 1 .510 

0 

. 298 

0.725 

0 

. 439 

0.939 

0 

.149 

0 4 02 

0 

. 435 

Stages  run  2100 

£ 

r ror  = 

0 .2769 

Std.  dev. 

of  error5 

0.1786 

Para ne ter  Std 

dev .s  of 

estimates  est 

i nates 

- 1 .5  02 

0 

288 

0.709 

0 

. 406 

0 .944 

0 

.144 

0.479 

0 

. 40  1 

Stages  run  2400 

£ 

r r o r = 

0 .2607 

Std.  dev. 

of  error* 

0 . 1 683 

Paraneter  Std 

dev .s  of 

est  i nates  est 

i nates 

- 1 .5  05 

0 

. 265 

0 .7  06 

0 

. 378 

0 950 

0 

139 

O 

* 

-1 

0 

386 

Stages  run  2700 

£ 

r ror  = 

0 2390 

Std.  dev. 

of  error5 

0.154  1 

142 
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Parameter  Std.  dev  s of 
estimates  estimates 

-1.314  0 . 234 

0 .701  0 . 338 

0.952  0.136 

0.474  0.330 


Stages  run  3000  Error  * 0.2261 

Std.  dev.  of  error*  0.1371 

Std.  d ev  . s of 
est i nates 
0 . 234 
0.317 
0.129 
0 . 324 


Parameter 
est i «a te  s 
-1.503 
0 .693 
0 .936 
0.484 


r 


E 

► 


! 


i 

i 

i 

I 


i 


i 

i 
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Std.  dev.  of  error®  0.2184 


Parameter 
est i wanes 
- 1 538 
0 7 20 
0.975 
0 4 9 8 


S td  dev.s  of 
est i nates 
0 30  9 
0 412 
0.140 
0.388 


Stages  run  1500 
S t d . dev. 


E r ror  * 
of  error® 


0.227  1 
0.1342 


Parameter 
esn  p nates 
-1.521 
0 .7  03 
0 970 
0.512 


Std  dev  s of 
est i nates 
0 246 
0 . 324 
0 12  8 
0.307 


Stages  run  1800  Error  = 
Std.  dev.  of  error® 


0.1931 
0 1058 


Parameter 
est i nates 
-1.523 
0 6 98 
0 971 
0 5 12 


Std.  d ev . s of 
est i nates 
0.204 
0.271 
0 117 

0.251 


Stages  run  2100  Error  = 
Std  dev.  of  error® 


0.1809 

0.1055 


Parameter 
est  > nates 
- 1 5 05 
0.688 
0 975 
0.526 


Std.  dev.s  of 
est  p nates 
0.189 
0.267 
0 . 10  7 
0.234 


Stages  run  2400  Error  = 
Std.  dev  of  error® 


0 1650 
0 0998 


Parameter  Std.  dev.s  of 
estimates  est i nates 


- 1 

5 05 

0 

17  1 

0 

693 

0 

246 

0 

933 

0 

105 

0 

5 12 

0 

218 

Stages  run  2700 

E r r o r = 

0.1461 

Std.  dev 

of  error® 

0.0979 

145 
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- 1 5 i * 

U 
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0 *>  9 & 

c 

215 

0 « L4 

<■*. 

09  S 

0 5 0 0 

0 

192 

S t >i  c e S r G n 3 0 0 0 Error  = 0 I 3 7 £ 
J t d aev.  of  error=  C . 0 7 0 7 

dev .s  of 
st ; h 1 1 e s 
0 1 3 fc 
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0 095 
0 17  1 
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System  order  n = 2 


The  system  parameters  aretin  the  o r d e r a 1 < . . an  > bl  / . , bn  ) : 

-1  5 00 

0 700 
0 0 00 

1 0 00 
0 5 00 
0 000 


Nob i na l noi 

se  ! 

Gauss 

i a n 

M ea  n = 

0 . 00 

St  d 

dev 

. = 

1 Ot 

C o n t a n < n a 1 1 

i ng  noise: 

La  p l a c 

i a n 

Mean* 

0 . 00 

St  d 

dev 

. = 

3 It 

C on  t am  . n a 1 1 

on  level  = 

0 

1 0 

The  Hampel 

parameters 

are: 

si  = 1 

00 

s2 

= -0 

50 

d 1 =>0 

2 5e  0 1 

d 2 

= 0 

45  e 

0 1 

ucutof  s 0 . 9 5 e 01 


i 


Stages  run 

300 

E r r o r = 

0 

4293 

Std  . 

dev. 

of  error= 

0 

3 185 

Parameter 

Std 

dev ,s  of 

est i hates 

est i nates 

- 1 .577 

0.444' 

0.811 

0 660 

0.956 

0.227 

0.408 

0.653 

Stages  run 

600 

E r r o r = 

0.3795 

Std  . 

dev. 

of  error= 

0.3367 

Parameter 

Std 

d ev . s of 

est i mates 

est i mates 

-1.551 

0.447 

0 774 

0 627 

1 0 04 

0.207 

j 

0.444 

0.617 

* i 

1 

Stages  run 

900 

E r r o r = 

0.2621 

1 Std. 

dev. 

of  error= 

0.1810 

i 

Parameter 

Std 

d ev . s of 

i 

estimates 

est i mates 

- 1 5 35 

0 295 

0 6 97 

0.398 

* 

0 9 92 

0.172 

,t 

0.515 

0 358 

X 

l 

1 ] 

i 

Stages  run 

1200 

E r r o r = 

0.2063 

147 


1 
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Parameter  Std  dew  s of 


est  i 

nates 

est  i 

i nates 

- 1 

324 

0 

136 

0 

720 

0 

199 

0 

9 99 

0 

080 

0 

478 

0 

169 

Stages  run 

3000 

Error  = 

Std  . 

dev. 

of  error= 

Paraneter 

Std 

dews  of 

e st i na  te  s 

est i na  te s 

-1.522 

0 116 

0 706 

0 17  1 

1 0 02 

0 078 

0 489 

0 149 

0 119  1 

0 061b 
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System  ordir  n = 


The  system  parameters  are( in  the  orderal 
- 1 300 

0 7 00 
0 0 00 

1 0 00 
0 5 00 
0 0 00 


< an  < b 1 / 


bn) 


Nominal  n 0 i 

58  i 

Gauss 

1 a n 

C on t am  inati 

ng  no 

i 3 e : Gauss 

i a n 

Contain  inati 

on  level  * 0 

05 

The  Hampel 

parameters  are: 

Stages  run 

300 

Error  = 

1 .2217 

Std  . 

dev 

of  error5 

2.4015 

Parameter 

Std 

d ev  3 of 

e s t i mates 

est i mates 

-2  .3  82 

3 175 

1.270 

3 039 

1 0 78 

0 326 

1 

O 

*■* 

2 836 

Stages  run 

600 

E r r 0 r 3 

0.9287 

Std  . 

dev. 

of  error= 

1 .1374 

Parameter 

Std 

d e v s of 

e s t i mates 

est 1 mates 

- 1 323 

1 370 

0 9 80 

2 104 

1.110 

0 223 

0 069 

1 . 526 

Stages  run 

900 

E r ro  r = 

0 6420 

Std. 

dev. 

of  error3 

0 6132 

Parameter 

Std 

dev .s  of 

est i nates 

est i mates 

-1.385 

0.733 

1 1 7 a 

1.002 

1 0 81 

0.183 

0 0 29 

0.968 

Stages  run 

9 * 

1200 

Error  3 

0.5569 

Plean=  0.00 
ft  ea  n=  0.00 


si  = 100 
dl  3 0 . 1 7e  38 


Std.  dev  . = 100 
Std . dev  . 3 10  00 


s2  = 1 00 
d 2 3 0 1 7 e 38 


ucutof  3 0 . 1 7 e 38 


150 


f '.dt 


A 
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Para ne ter 

Std 

dews  of 

est i nates 

est 

i nates 

- 1 609 

0 

222 

0 £25 

0 

. 309 

0 993 

G 

.115 

0 408 

0 

. 339 

Stages  run 

3000 

c 

r ror  = 

Std  . 

dew. 

of  error= 

Paraneter 

Std 

d ew . s of 

est i nates 

est 

i nates 

-1  5 SO 

0 

211 

0.797 

0 

294 

0 9 94 

0 

.115 

0 426 

0 

.316 

0 2169 
0.1395 
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System  order  n * 2 


The  system  parameters  ore(  in  the  order  a 1 < . . « an < b 1 < . . > bn  ) : 

-1.500 

0 700 
0 .000 

1 .000 
0 500 
0.000 

Nominal  noise:  Gaussian  Mean®  0.00  Std.  dev. * 1 00 

Contaminating  noise:  Gaussian  Mean*  0.00  Std.  dev. “10  00 

C on t am i n a t i on  level  * 0.03 

The  Hampel  parameters  are:  si  » 1.00  s2  * 0.00 

dl  > 0.20s  01  d2  * 0.1?e  38 

ucutof  = 0 . 1 7e  38 


Stages  run  300  Error  * 0.3110 

Std.  dev.  of  errors  0.3308 


Par ameter 
estimates 
-1.772 
1 .064 
1.122 
0.191 


Stag*3  run 
Std. 

Parameter 
est  i mates 
-1.649 

0 .896 

1 .099 
0 .277 


Stages  run 
Std 

I 

| Parameter 

est  i nates 
-1.668 
0 858 
1.059 
0 .296 


Stages  run 


Std.  d ev . s of 
estimates 
0 . 600 
0 . 932 
0 . 278 
0.711 


600  Error  * 03420 

dev.  of  error*  0.2418 

Std.  d ev  . s of 
est i nates 
0.317 
0 . 508 
0 . 193 
0 . 429 


900  Error  = 0.3069 

dev.  of  error*  0.2161 

Std.  dev  . s of 
est i ma  tes 
0 . 297 
0 . 420 
0 . 139 
0 416 


1200  Error  * 0.2603 

1S3 


r 
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S t d dev. 

of  error= 

0.1256 

Parameter  Std 

dev .s  of 

esc  iMtes  e st  1 notes 

- 1 .61? 

0 236 

0 7 3? 

0 312 

1.034 

0.142 

0 335 

0 352 

Stages  run  1500 

Error  = 

0 .227  1 

Std.  dev. 

of  e rr 0 r = 

0.1397 

Parameter  Std 

d ev  . s of 

estimates  estimates 

- 1 .5  33 

0.208 

0 .772 

0 274 

1 .0  26 

0.152 

0.413 

0.349 

Stages  run  1800 

Error  * 

0 . 1957 

Std.  dev. 

of  error' 

0.1243 

Parameter  Std 

d ev  . s of 

estimates  estimates 

- 1 .555 

0.198 

0.745 

0 . 238 

1.011 

0.121 

0.457 

0.311 

Stages  run  2100 

Error  = 

0.1818 

Std.  dev. 

of  error' 

0.1249 

Parameter  Std 

dev .s  of 

estimates  estimates 

-1.570 

0.179 

0 .778 

0 . 233 

1.009 

0.102 

K 

0.427 

0 . 284 

Stages  run  2400 

Error  = 

0 . 1587 

m 5. 

Std.  dev. 

of  error= 

0 . 1207 

Parameter  Std 

d ev . s of 

estimates  estimates 

- 1 .554 

0.169 

0.74? 

0.220 

1 000 

0 103 

* 

0 4 57 

0 . 253 

Stages  run  2700 

E r ro  r * 

0 . 1 409 

Std  dev. 

of  error* 

0.1096 

[ 

154 

r .-..li  < S-A*  . >•  1 ■ ■» 

2 
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Paraneter 
esc  i nates 
- 1 552 

0 .753 

1 .004 
0.430 


Std.  dev  .s  of 
est  t notes 
0 . 142 
0 . 196 
0 090 
0 . 229 


Stages  run  3000  Error  = 0.1273 

Std.  dev.  of  error*  0.0841 


Parameter 
est i nates 
-1.543 

0 .725 

1 .002 
0 .467 


Std.  d ev  . s of 
est  mates 
0 . 125 
0.161 
0 . 087 
0.201 


J un 
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System  order  n - 2 


The  system  parameters  ar«(  in  the  orderali  . an<bl  < . . . bn  ) s 
-1 -500 
0.700 
0.000 
1 0 00 
0 500 
0 .0  00 


Nominal  n o i 

se  • 

G a us  s i an 

Mean3  0.00 

Std 

dev 

. 3 1 

. 00 

Contain  inati 

ng  no  is« i 

G a us  s i an 

tl  ea  n=  0.00 

St  d 

dev 

. 3 10 

. 00 

Contafi  inat: 

on  level  = 

0 05 

The  Han pel 

parameters 

are! 

si  3 1.00 
dl  3 0 . 1 5e  01 

ucutof  3 0 

3 £. 
d 2 

. 1 7e  38 

3 0 . 

3 0 

00 
1 7e 

38 

Stages  run 
Std. 


300  Error  3 
dev.  of  error3 


0 .4773 
0.4407 


Parameter 

Std. 

d ev  . s of 

e st i mates 

e st 

i mates 

- 1 739 

0 

. 537 

1 0 09 

0 

. 733 

1.147 

0 

.310 

0.187 

0 

637 

Stages  run 

600  E 

r ro  r = 

Std  . 

dev.  of  error3 

Parameter 

Std  . 

d ev  . s of 

e s t i nates 

e st 

i mates 

-1.646 

0 

.310 

U 

CO 

*5% 

CO 

0 

. 471 

1.112 

0 

.210 

0.275 

0 

395 

Stages  run 

900  E 

r ror  = 

Std 

dev  of  error3 

Parameter 

Std 

d e v . s of 

e s * i mates 

* 3 t 

i mates 

- 1 647 

0 

287 

0 861 

0 

396 

1 0 76 

0 

172 

0 2 80 

0 

. 396 

Stages  run 

1200  E 

r r o r 3 

0.3262 
0 .2276 


0.3042 
0 1995 


0 261  7 


1 56 
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Std.  dev.  of  error® 


P ar  a ne  te  r 

Std . dev  . s of 

estimates 

est i nates 

-1.609 

0 . 223 

0.798 

0 . 307 

1.052 

0 . 136 

0 .339 

0 . 350 

Stages  run 

1300 

Error  = 

Std. 

dev. 

of  error® 

Puraneter 

Std 

dev ,s  of 

est i nates 

est  i nates 

-1.590 

0.213 

0 .772 

0.231 

1 .042 

0 162 

0 .391 

0 342 

Stages  run 

1800 

E r ro  r * 

Std  . 

dev  . 

of  error® 

Paraneter 

Std 

dev  s of 

est i nates 

e st i na  te  s 

-1.573 

0.209 

0 .737 

0 . 234 

1 .026 

0 131 

0.434 

0.319 

Stages  run 

2100 

Error  ® 

Std . 

dev. 

of  error® 

Par aneter 

Std 

dev . s of 

e s t i na  te  s 

e st i na  te  s 

-1.578 

0 . 189 

0.772 

0 . 234 

1 .023 

0.113 

0.408 

0 . 303 

Stages  run 

2400 

Error  = 

Std . 

de v . 

of  error® 

Paraneter 

Std 

. dev  s of 

e st i na  te  s 

e st i na  te  s 

-1.357 

0 . 172 

0 74? 

0 . 226 

1 013 

0.113 

0 439 

0 . 270 

Stages  run 

2700 

Error  ® 

Std. 

dev. 

of  error® 

0 . 1242 


0 .2334 
0.124? 


0 . 2058 
0.1233 


0.1934 

0.1291 


0 .1639 
0.1283 


0.1462 

0.1139 
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Paraneter 

S td  dev . s of 

estimates 

«st i Kates 

-1.350 

0.142 

0 . 750 

0.201 

1.016 

0 . 098 

0 437 

0 . 244 

Stages  run 

3000  Error  « 

0 

$ t d 

de v . of  error® 

0 

Parameter 

S td  dev  i of 

esc i nates 

est i nates 

-1.546 

0.136 

0 723 

0 . 164 

1.012 

0 . 094 

0.433 

0.216 
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The  system  parameters  tr«(  in  the  order  a 1 < . . » an > b 1 > 
-1.500 

0 .700 
0.000 

1 000 
0 .500 
0 .000 


Nominal  noise:  Gaussian 

Contaminating  noise:  Gaussian 

C on t an  i n a t i on  level  3 0.05 

The  Hanpel  parameters  are: 


Dean*  0.00 
Mean*  0.00 


1 00 

0 lOe  01 


Std.  de  v . = 1.00 
St  d . de  v . = 1 0 00 


s 2 3 0 00 
d 2 3 0 17e  38 


uc u to f 3 0 . 1 7*  38 


Stages  run 

300 

Error  3 

Std . 

de  v . 

of  error3 

Par aneter 

Std 

. d *v  . s of 

e st  i na  te  s 

est i nates 

-1.758 

0 . 568 

1 002 

0.781 

1.156 

0 344 

0 .083 

0 . 778 

< 

Stages  run 

600 

Error  3 

Std . 

de v . 

of  error3 

Parameter 

Std 

dev . s of 

estimates 

e st  i na  te  s 

- 1 665 

0.310 

0 .826 

0 . 438 

1.123 

0 . 246 

0 .248 

0 . 368 

Stages  run 

900 

Error  3 

Std . 

de  v . 

of  error3 

1 Parameter 

Std 

. dev . s of 

e st  i ma  te  s 

est i nates 

-1.638 

0 . 264 

0 8 78 

0 . 398 

1 100 

0 . 209 

0 .228 

0 . 388 

Stages  run 

1200 

Error  3 

0 .3141 


3 0.2827 


1 


^ . -* 


r 


J un 
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Std  . dev.  of  error* 


0.1331 


P ar  a ne  ter 

Std  d ev . s of 

e s t i nates 

e st  i na  te s 

-1.820 

0 . 230 

0 .829 

0.319 

1 .079 

0.191 

0 286 

0 . 375 

Stages  run 

1500 

Error  ■ 

Std  . 

dev  . 

of  error* 

P ar  a ne  te  r 

Std 

dev . s of 

e st i nates 

e st  i na  te  s 

-1  .393 

0.211 

0.818 

0 . 280 

1 .066 

0 . 187 

0.316 

0 . 372 

Stages  run 

1800 

Error  * 

Std  . 

dev. 

of  error* 

P ar  a ne  te  r 

Std 

dev . s of 

e st i na  te  s 

est i nates 

-1 .576 

0.194 

0 791 

0 . 249 

1 .054 

0 . 162 

0 .356 

0.340 

Stages  run 

2100 

Error  = 

Std  . 

dev. 

of  error* 

P ar  a ne  te  r 

Std 

. dev  .s  of 

est i na  te  s 

est i notes 

-1.608 

0 . 205 

0 .781 

0 . 240 

1 .047 

0 . 142 

0 .351 

0.321 

Stages  run 

2400 

Error  « 

Std. 

dev . 

of  error* 

P ar  a ne  te  r 

Std 

d ev . s of 

est i na  te  s 

est i nates 

- 1 .380 

0 . 184 

0 .763 

0 228 

1 .038 

0.137 

0 377 

0 . 296 

Stages  run 

2700 

Error  ■ 

Std 

dev 

of  error* 

0 .2584 
0 .1509 


0 .2270 
0.1381 


0.2132 

0.1434 


0 . 1 885 
0 . 1421 


0 1898 
0.1303 
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Systen  order  n = 2 


L2 


Th*  systen  parameters  ar«( in  th*  orderal 
-1.300 
0 .700 
0.000 
1 .000 
0 .300 
0 .000 


an  < b 1 


bn) 


Noninat  noise:  Gaussian 

Con t an i n a t i ng  noise:  Gaussian 


Contan i nat 

ion  level  * 

0 

.03 

T he  Ha  np  e l 

paraneter*  are : 

Stages  run 

300 

Error 

S 

0 .7463 

Std 

. de  v . 

of  error* 

0 .8383 

Paraneter 

Std 

dev  . s 

of 

est  i nates 

est  i nates 

-1  .826 

1 . 024 

0 .537 

1519 

1.124 

0 . 336 

0 .328 

1 . 232 

Stages  run 

600 

Error 

3 

0 .3207 

Std 

. de  v . 

of  error* 

0 .3520 

Paraneter 

Std 

d *v  . s 

of 

est i nates 

est i nates 

-1.778 

0 . 572 

1 .042 

0.771 

1.132 

0 . 303 

-0.037 

0 . 883 

Stages  run 

900 

Error 

3 

0.4167 

Std 

dev . 

of  error* 

0 .3035 

Par  an*  ter 

Std 

. dev . s 

of 

est  i na  t* s 

e st  i na t* s 

-1.769 

0 . 385 

0 923 

0.313 

1 106 

0 . 240 

0 077 

0 . 333 

Stages  run 

1200 

Er  ror 

3 

0.3714 

- * , > 

dean*  0.00 
(lean*  0.00 


Std.  dev  .«  1.00 
Std.  dev. *10. 00 


si  • 1.00 
dl  * 0.70*  00 


s 2 » 0 .00 
d 2 = 0.17*  38 


ucutof  * 0.17*  38 
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Std . 

dew.  of  »rror= 

0.2862 

Par aneter 
ist i natts 
-1.702 

0 912 

1 .096 
0.128 

Std  . dew  . s of 
est  i nates 
0.361 
0 . 429 
0 . 230 
0 . 526 

Stages  run 
Std. 

1300  Error  * 

dew.  of  error* 

0 .3384 
0 .2943 

P ar  a ne  t*  r 
* s t i na  te  s 
- 1 .707 

0 836 

1 .078 
0.181 

J 

Std  . dew . s of 
est i nates 
0.414 
0 . 393 
0 . 209 
0 . 307 

1 

Stages  run 
Std . 

1800  Error  * 

dew.  of  error* 

0 .2940 
0 .2402 

P ar  a ne  ter 
est i nates 
-1.637 
0 .860 
1 .071 
0.218 

Std  . dew  . s of 
est i nates 
0 . 258 
0 . 338 
0 . 194 
0 . 462 

1 


Stages  run 
Std. 

2100 
dew  . 

Error  * 

of  error* 

0.2814 
0 .2331 

Paraneter 
e st i na  te  s 
-1.673 

0 .827 

1 .063 
0 .233 

Std  . dew  . s of 
est i nates 
0 . 292 
0 . 343 
0 . 174 
0 . 423 

1 

Stages  run 
Std. 

2400 
dew  . 

Error  * 

of  error* 

0 .2464 
0 2026 

Par aneter 
est i nates 
- 1 639 
0 808 
1 038 
0 264 

Std.  dew . s of 
est  i nates 
0 . 260 
0 . 273 
0 . 169 
0 . 380 

Stages  run 

Std . 

2700 
de v . 

Error  * 

of  error* 

0 .2349 
0.1919 
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Parane  ter 

S td . dew  s of 

est i nates 

est i nates 

-1  .615 

0 . 225 

0 .623 

0.271 

1 .059 

0 . 163 

0 .269 

0 . 362 

Stages  run 

3000 

Error  * 

Std 

dev  . 

of  error* 

Parameter 

Std 

dew . s of 

est i nates 

e st  i na te s 

-1.614 

0 . 225 

0 .804 

0 . 260 

1 .054 

0 . 154 

0 .285 

0 . 339 

0.2218 
0 .1826 
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'J  'j 


ujter.  o r J ■> r n = 


• suite  ’■> 

■:  too 
0 ’00 
0 0 0 0 


Sirihet  ers  a r e < i n the  o r d e r a 1 / » a n , b 1 > .bn'1  • 


0 5 0 
0 0 0 


Mor.  1 na 

L n 0 1 

S-?  ? 

G a us  s 1 an 

Mean®  0 00 

St  d 

dev.®  1 

C 0 r<  r . Ft 

. rt  «i  t . 

n 3 no  * 59 : 

Gauss i an 

Mean®  000 

St  d 

dev  ® 1 0 O'. 

O *•  t « f 1 

. n a t • 

0 n level  = 

0 05 

7 h * H •:  : 

■ pel 

parane^ers 

are! 

si  = 1.00 

s 2 

= - 0 27 

dl  = 0 1 2e  G ~l 

d 2 

= 0 3 5 e 0 1 

ucuto*’  = 0 . 

7 9 e 01 

Stages 

r »;  n 

3 0 0 E r r o r 

= 05503 

; t d 


P ar  a r.e  ter 
e;ti  r-i  a t e s 
- 1 7 6 ■* 

1 018 
: : '0 
i*  • ; n 


d e v . o 
S t d 


error®  0 5041 


dev  .s  of 
s t i nates 
0 620 
0.021 
0.329 
0.729 


run  6 G 0 Error  = 
Std  dev  of  error® 


0 3 78  2 
0 4 00  2 


P fi  *'  4 ••  $ **r 

•? I " : i • i ‘5  i 


0 2 60 


■stages  r u n 
3 t d 

Par  a r.e  t e r 
esc : nates 
- 1 6 3 9 
0 i 6 0 
G 9 0 
0 _ ~ 1 


; d . dev . s of 
est i nates 
0 47  0 
0.726 

n ^ ^ -» 

*J  4.^.  I 

0.531 


900  Error  = 
dev  of  error® 

std  d e v . s of 
est i nates 
0 35  9 

a 50  i 

0 19  1 
0 44  3 


0 3275 
0.2703 


t a g s run  12  00  Error 
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J ur 


Std  . 

dev. 

of  error1 

0 1577 

Parjueter 
est i hates 
- 1 .626 
0.789 
1.068 
0 338 

Std  dev.s  of 
est  i hates 
0.268 
0.350 
0.171 
0.363 

Stages  run 
Std. 

1500 

dev. 

Error  = 

of  error= 

0.2450 

0.1371 

Paroheter 
est i hates 
“1  5 86 

0 770 

1 .0  54 
0.370 

Std  dev.s  of 
est i hates 
0 . 220 
0 292 
0.169 
0.346 

Stages  run 

1800 

E r r o r = 

0 221  6 

Std  . 

dev. 

of  error= 

0.1276 

Para  he  ter 

Std 

dev.s  of 

est i nates 

e s t i ha  te  s 

- 1 558 

0.206 

0 7 fc  0 

0.266 

1 0 44 

0.148 

0 401 

0 . 327 

Stages  run 

2100 

Error  = 

0.2119 

Std  . 

dev. 

of  error® 

0.1434 

Parameter 

Std 

dev.s  of 

est i hates 

e 

s t i hates 

” 1 .577 

0.213 

0.772 

0 27  1 

1 0 38 

0.128 

0 3 85 

0.317 

Stages  run 

24  00 

E r r o r = 

0 

1856 

Std  . 

dev. 

0 f 

error* 

0 

1373 

Paroheter 

Std 

dev.s  of 

est i nates 

e 

St  i 

nates 

- 1 551 

0 

18  1 

0 752 

0 

260 

1 030 

0 

125 

0.4;5 

0 . 

289 

Stages  run  2700 

Error  = 

0 1698 

Std  dev. 

of  errors 

0.1195 

166 


F 


k 
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i 
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System  order  n = 2 


The  =ysto n parameters  a r e < i n the  orderal<  . . ,an,bl 
-1  5 00 

0 7 00 
0.000 

1 0 00 
0 .5  00 
0.000 


nominal  noise: 


G a us  s i an 


PI  ea  n = 0.00 


Std.  de  v . = 1.00 


Contaminating  noise:  Gaussian 


PI  ea  n=  0 .00 


Std  . dev  *10 . 00 


Contamination  level 


The  Hampel  parameters  are: 


Stages  run  300  Error  = 0.5347 

Std.  dev.  of  error=  0.5141 


si  = 1.00 
dl  = 0 . 25e  01 


s 2 =-0.50 
d 2 = 0 . 45e  0 1 


ucutof  = 0 . 9 5 e 01 


Parameter 
e s t i nates 
- 1 .764 
1 OSS 
1.124 
0.179 


i td . dev  . s of 
e st i nates 
0.623 
0.919 
0 . 278 
0 . 737 


Stages  run  600  Error  = 0.3368 

Std.  dev.  of  error=  0.3118 


Parameter 
e s t i nates 
- 1 .663 

0 S •*  6 

1 0 34 
0.323 


Std.  dev.s  of 
est i nates 
0.417 
0.564 
0.183 
0 . 475 


Stages  run  900  Error  = 0.3111 

Std.  dev  of  error=  0.2573 


P a r a n e t e r 
est : nates 
- 1 647 

0 3 47 

1 0 49 
0 3 26 


Std  dev.s  of 
est i nates 
0.364 
0 45  7 
0 . 159 
0 456 


Stages  run  1200  Error 


1 


M 


1 


f 

I 


« 

i 


o ur. 

1 1 

L 3 

:39  no.se  13  Page  2 

3 t 

•j  ijjv  of  error1  0 . 1 1 S 7 

P i -i 

■ 1 * 

r.  £ r 

i t d d e v s of 

» ■ ■ 

" A ■£ 

e s t i n a t e s 

- ‘ 

5 

9 o 

0 2 4 6 

0 

r • 

0.293 

■ 

* c 

0 13  3 

- 

; 4 

0 34  3 

S r 

9 * - 

run 

15  0 0 £ 

r 

r o r = 

0 

2 156 

S t d 

d e v o 

f 

error= 

0 

.1337 

Par 

a r.  r 

• e r 

S t d . 

d 

e v s of 

err 

1 h*i 

r,  £ 3 

e s t 

i 

nates 

- 

• ~ 

- - 

*j 

22  1 

0 

0 

264 

* j 

• ■* 

o 

145 

0 4 

a 9 

0 

33  3 

3 i u 

j **  i r u n 

ISO 

0 

E r r o r 

= 

0 13  4 0 

3 t d . 

d * 

v . 

of  erro 

r = 

0 . 1 297 

P a r 

•i  i t 

$ 

td 

a e v s 

of 

?r. 

: ft  d * 5 3 

e 

s t i nates 

- 

1 5 7* 

G 20  5 

0 7 21 

0.243 

1 0 04 

0 122 

0 4 39 

0 312 

3 t u * 

5 run 

2100  Error  = 

0 

1 73  9 

St  d . 

dev  of  error1 

0 . 

17  10 

k j k 'j 

P i r a n 

3 f 

S t d dev  s of 

r?  I " ; r * 

*1  v $ • 

e 3 1 i nates 

- « 

Z 'j  o 

0 . 19  2 

0 

■’Sfl 
V »j 

0.245 

i 

0 C 5 

0 . 10  0 

0 

4 Z i 

0 23  4 

3 t g «?  £ 

run 

24  0 0 E 

rror  = 

0 1613 

3 t d . 

dev  o 

f e r r o r = 

0 119  1 

P i r*  «i  r 

ter 

3 td 

dev  s of 

* 5 t : ft  *1 

t e • 

e s t 

i nates 

«.  « c 

■ *% 

“*  w 

0 

175 

•j  ^ 

3 

J 

232 

r ^ 

5 

Q 

100 

0 4 

" Cl 

0 

250 

Stage*  ran  2700 

Error  = 

0 14  3 9 

% r.  d d e v 

of  error* 

0 109" 
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Paraheter 

Std 

d e v . s of 

f sc : hates 

e 

s t : nates 

-1  5 45 

0 153 

G "'34 

0.208 

1 GOG 

0 088 

0 4 8? 

0.228 

Stages  run 

30GO  Error  = 

0.1306 

Std. 

dev.  c f 

error5 

0 .0836 

Paraneter 

Std  dev.s  of 

esti nates 

e S t : 

notes 

- 1 526 

0 

130 

0 7 17 

0 . 

17  1 

0 V ? £ 

0 

086 

0 4 82 

0 . 

203 

170 
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S y s t e order  n = 


The  system  parameters  a r e ( i n the  ordera  1 , . , i an / b 1 . . . , bn  / ; 

-1.50C  \ 

0.7  05 

-0  100  \ 

0 0 05 
0 0 48 
-0  008 


Nominal  no.se' 

Gauss . an 

Mean* 

0 . 00 

Std 

dev  = 1 

00 

Contaminating  no.se: 

La  p 1 a c i an 

Hear.* 

0 00 

Std 

dev  =10 

00 

Contamination  level  = 005 

The  Hampel  parameters  are:  31*100  s 2 = 1 0 0 

dl  * 0 17e  33  d2  » 0 l?e  38 

ucutof  * 0 l 7 e 38 


Stages  run  300  Error  = 19036 

Std.  dev  of  error*  1 1228 


Parameter 

Std  . 

dev .s  of 

e s t i nates 

e s t 

i mates 

- 0 4 08 

0 

. 92  1 

-0  255 

0 

. 931 

-0.156 

0 

. 438 

0.541 

1 

354 

0.542 

1 

. 75  1 

0 . 390 

1 

. 530 

Stages  run 

600  Error  = 

2.2683 

Std 

dev  of  error* 

3.7852 

Parameter 

Std 

dev.s  of 

e s t i mates 

e s t 

i mates 

-0.138 

1 

533 

-0.171 

0 

. 824 

-0  195 

0 

. 350 

0 j 1 5 

4 

372 

0 . 2S6 

4 

063 

0 259 

3 

497 

Stages  run 

900  Error  = 

3.6052 

Std.  dev  of  error*  105981 


Parameter  Std.  dews  of 

estimates  estimates 

-0  497  1 939 

- 0 326  1.405 

-0  218  0 797 


171 
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2 

657 

1 1 

363 

2 

4 4 4 

10 

535 

2 

.0  73 

8 

988 

Stages  run 

1200 

E r ro  r = 

1 .7120 

Std 

dev. 

of  # rr  o r = 

1 .3053 

Para neter 

Std 

dev  s of 

est  i nates 

est . nates 

-0.386 

0.762 

-0  .246 

0 . 527 

-0.144 

0.319 

0 .848 

1 796 

0 .783 

1 . 675 

0.667 

1 . 435 

Stages  run 

O 

O 

hi 

Error  = 

3 4263 

Std  . 

dev 

of  error® 

7 .3031 

Para  Me  ter 

Std 

dev .s  of 

est  i nutes 

est  i nates 

- 1 .2  66 

3 . 479 

-0.928 

2.894 

-0  422 

1.038 

2 .902 

7.383 

2 693 

6 869 

2.280 

5 . 787 

Stages  run 

1800 

E r ro  r = 

1 .5420 

Std  . 

dev. 

of  error® 

0.8114 

Paraneter 

Std 

dev ,s  of 

est i ntces 

e st  i na  te  s 

-0  242 

0.517 

-0.162 

0.336 

-0.089 

0.189 

0.688 

1.360 

0.631 

1 . 263 

0 530 

1 . 085 

Stages  run 

2100 

E r r o r = 

1 .8256 

Std 

dev 

of  error® 

10611 

Paraneter 

Std  dev  . s 

est i na  te  s 

e st  i na  te  s 

-0  4 16 

0.553 

-0.270 

0 . 363 

-0  147 

0.205 

1 114 

1 662 

1 027 

1 54  1 

0 870 

1 322 
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Stag 

•?  s run 

2400  Error  = 

Std 

dev  of  error= 

Paraneter 

Std 

d e v . s of 

e s t i 

nates 

e s t 

i nates 

- 1 

2 68 

4 

. 248 

-0 

8 12 

2 

. 320 

-0 

a57 

1 

675 

3 

3 85 

1 1 

564 

3 

.330 

10 

. 789 

2 

. 84? 

9 

. 275 

Stages  run 
Std 

2700  Error  = 

dev.  of  error* 

Paraneter 

Std.  dev  . s of 

e s t i 

nates 

est i nates 

-0 

5 39 

1 291 

-0 

356 

0 875 

-0 

.2  14 

0 . 522 

0 

. 9S6 

3.149 

0 

913 

2.940 

0 

779 

2.529 

Stages  run 

3000  Error  = 

Std  . 

dev.  of  error* 

P a r a m e t e r 

Std.  dev  . s of 

e s t i n a t « s 

est  i nates 

-0.143 

1 . 073 

-0  0 95 

0.651 

-0.052 

0.347 

0.471 

3 . 875 

0 . 425 

3 . 59  5 

0 343 

3 . 077 

4.4335 

11.1638 


2.3211 

2.2807 


2 851  0 
2 .6604 
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Systen  order  n * 3 


Th*  sy  s t e n paroneters  »rt(  in  th*  ordtrtli  . . >tn<bl 
- 1 3 00 


-0  008 

Non i no ' noise:  Gaussian 

C on t an . n a t i ng  noise:  Laplacian 
C on t an i n a t i on  level  * 0 03 

The  Hanpel  paraneters  are: 


Mean*  0.00 

H ea  n=  0.00 


si  « 1.00 
dl  = 0 . 20e  0 1 


bn)  : 


St  d . dev.*  1.00 
St  d . dev  *10.00 


s 2 * 0 00 
d 2 * 0 . 1 7 e 38 


ucutof  * 0.17e  38 


Stages  run 

300 

Error  * 

Std . 

dev 

of  error* 

Paraneter 

Std.  d ev . s of 

est i nates 

est i nates 

-7.187 

37 . 782 

-4.364 

22.710 

-2  .406 

12 . 237 

18.817 

100 . 064 

18.107 

96 . 360 

14.413 

76.581 

Stages  run 

600 

Error  = 

Std. 

dev 

of  error* 

Paraneter 

Std.  dev . s of 

e s t i na  te  s 

est i nates 

-0.477 

0 763 

-0.313 

0 . 423 

-0.174 

0 . 192 

1 .381 

3.174 

1 .276 

2 . 943 

1 .083 

2.512 

Stages  run 

900 

E r r o r * 

Std  . 

dev 

of  error* 

Paraneter 

Std.  dev . s of 

est i na  tes 

est i nates 

-0.371 

0 . 692 

-0.239 

0 . 434 

-0  146 

0 . 240 

1 .8467 


2.1003 


J u n 7 2 0^1 

2 no 
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1 

1 0 30 

2 . 39 S 

0 9 4 7 

2 23  2 

0 801 

1 912 

Stages  run 

1200 

E r ror  = 

2.3946 

Std  . 

dev. 

of  error= 

3.1749 

Parameter 

Std 

dev.s  of 

est  i nates 

est  i nates 

- 0 181 

1 . 483 

-0.118 

1.054 

-0  071 

0.631 

0.598 

3.824 

0 5 46 

3 57  1 

0 454 

3 075 

Stages  run 

1500 

E r ro  r = 

1 444  6 

Std  . 

dev 

of  error* 

0 4982 

Par  a ne ter 

Std 

dev.s  of 

est  i nates 

est  i na  te  s 

1 

O 

f'J 

iv> 

0 . 434 

-0  168 

0 . 295 

-0  103 

0 . 169 

0 5 63 

1116 

0.515 

1.045 

0.430 

0.900 

Stages  run 

ISO  0 

Error  = 

5 .3562 

Std  . 

dev 

of  error* 

16.4406 

Paraneter 

Std 

dev.s  of 

est  i nates 

e 

st i nates 

0 .3  20 

4.058 

0 0 35 

2 . 045 

-0  0 43 

0 . 737 

-2  370 

17.788 

-2  2 02 

16.501 

- 1 . 8S2 

14.078 

Stages  run 

2100 

Error  = 

2.4394 

Std 

dev. 

of  error* 

2.7189 

Parameter 
est  i nar.es 
-044-3 
-0  132 
-0  2 4? 

1 6 58 
l 5 36 
1 3 09 


Std  dev.s  of 
est i nates 
1266 
0.331 
0.491 
3.203 
2 989 
2 575 
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Stages  run  2400 
Std  dev 


Error  ■ 
of  error* 


1 8752 
2.4158 


P ar  a me  te  r 

Std  d ev  . s 

of 

e st i nates 

estimates 

-0.524 

1.082 

-0.341 

0 . 703 

-0.185 

0 . 375 

1.377 

2 642 

1 .274 

2 466 

1.082 

2 . 124 

Stages  run 

2700  Error 

C 

Std  . 

dev.  of  error® 

Parameter 

Std.  d ev . s 

of 

est i nates 

estimates 

0.194 

5.640 

0 053 

3.481 

-0  039 

1 . 672 

-1.893 

20  . 524 

-1  763 

19  095 

-1.510 

16.340 

Stages  run 

3000  Error 

S 

Std 

dev  of  error® 

Parameter 

Std.  dev . s 

of 

e st i na  te  s 

est i na  te  s 

0 .007 

0 . 845 

0.010 

0 . 553 

0.007 

0 . 305 

-0.002 

2.421 

-0.013 

2 . 256 

-0  026 

1 936 

5 .9423 


1.9843 
1 .8898 
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System  order  n 


The  system  parameters  a r e ( i n the  orderal 
-1  5 00 
0.705 
-0  100 
0 0*5 
0.048 
-0  0 08 


, an . b 1 


b n > 


Honinal  noise:  Gaussian 

Contm  . noting  noise:  Laplacian 
Contamination  level  = 005 

The  Hampel  parameters  are: 


PI  ea  n=  0.00 
H ea  n=  0.00 


Std.  dev.=  1.00 
Std  de  v = 10  ’0 


Stages 


run 

Std 


Parameter 
est  i mates 
- 0 .9*0 


-0  571 
- 0 2 '3  7 


348 
1 74 
S *8 


Stages  run 
Std 


Parameter 

estimates 

-0.738 


-0 

-0 

2 

2 


>8 
458 
235 
578 
3 80 
0 33 


Stages  run 
Std 


Parameter 
est i m a t e s 
1 8 18 
1 0 45 
0 5*7 


si  = 1.00 
dl  = 0 . 1 5e  01 


s2  = 0 00 

d 2 = 0 1 7 e 3 8 


ucutof  = 0.17e  38 


300  Error  = 
dev.  of  error1 


3 0670 
7.9708 


Std.  dev  . s of 
est i mates 
2.909 
1.504 
0.480 
8 373 
7 753 
6 . 723 


600  Error  = 2.9400 

dev.  of  error=  45393 


Std  d ev  s of 
est i mates 
1.028 
578 
260 
970 
615 
933 


900  Error  = 7 . 1440 

dev  of  error=  20.7792 


Std  dev.s  of 
est i nates 
9 42  1 
6 269 
3 579 
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-4  .226 

21  237 

-3  949 

19  842 

-3.406 

17.102 

Stages  run 

1200 

Error  = 

1 .6158 

Std 

dev. 

of  »rror= 

0 .9473 

P ar  a ne  te  r 

Std 

. dev .s  of 

est i nates 

est i nates 

-0  392 

0 . 492 

-0.268 

0 . 347 

-0.150 

0 . 190 

0 99? 

1 . 394 

0 9 19 

1 300 

0 778 

1.114 

Stages  run 

1500 

E r r o r = 

1.5217 

Std  . 

dev. 

of  error” 

0 .9725 

Para ne ter 

Std 

dev .s  of 

e st i na  te  s 

e st i na  te  s 

-0 .279 

0 . 438 

-0.181 

0 . 283 

-0.100 

0 . 155 

0.798 

1.421 

0 733 

1 . 322 

0.615 

1.131 

Stages  run 

1800 

Error  = 

2 .367? 

Std  . 

dev. 

of  error” 

4 .4900 

P ar  a ne  te  r 

Std 

dev .s  of 

est i nates 

e st  i na  te  s 

-0  .584 

1 684 

-0.388 

1.119 

-0  2 17 

0.612 

1 630 

4 . 825 

1 .509 

4 . 497 

1 .283 

3 . 866 

Stages  run 

2100 

Error  = 

1 4430 

Std 

dev. 

of  error” 

0 .4389 

P ar  a ne  te  r 

Std 

dev  s of 

e s t i nates 
-0  2 13 
-0.145 
-0  086 
0 506 
0 462 
0.384 


est i nates 
0.406 
0 . 266 
0.149 
1.114 
1.041 
0 . 897 
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Stages  run  2400  Error  s 2.3331 


Std 

dev.  of  error= 

4.1726 

Parameter 

S td . dev  .s  of 

est i nates 

est  i mates 

- 0 13  7 

1.883 

-0  0 ?2 

1.214 

-0  053 

0.641 

0 4 88 

4.731 

0 4 24 

4 463 

0 343 

3.845 

Stages  run 

2700  Error  = 

2 6 136 

S t d 

dev.  of  errors 

2.4616 

Parameter 

Std.  dev  .s  of 

est i nates 

est i nates 

-0.837 

0 . 993 

-0.419 

0.631 

-0  236 

0 . 338 

1.737 

3.147 

1 627 

2 . 932 

1 3 34 

2.512 

Stages  run 

3000  Error  = 

1 4853 

S t d 

dev.  of  error= 

0 5839 

Parameter 

Std  dev . s of 

est i mates 

estimates 

-0  2 90 

0.455 

-0  195 

0.311 

-0111 

0 186 

0.715 

1.148 

0 6 37 

1 073 

0 531 

0.923 
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System  order  n 3 3 


* 

i 


The  system  parameters  are(  in  the  orderal  < . . > an  >b  1 < . . i bn  ) 
-1.300 
0 .705 
-0.100 
0 . 065 
0.048 
-0.008 

Nominal  noise:  Gaussian 

Contaminating  noise:  Laplacian 
Contamination  level  * 0.05 

The  Hampel  parameters  are: 


Stages  run 

300 

Error  3 

1 .4272 

Std . 

dev  . 

of  error3 

0 .4274 

Parameter 

Std 

d ev  . s of 

est i nates 

estimates 

-0 .275 

0.412 

-0.209 

0 . 234 

-0.102 

0 . 164 

0 .802 

0 . 922 

0 .734 

0 . 852 

0.621 

0 . 727 

Stages  run 

600  Error  3 

1.4816 

Std . 

dev . of  error3 

0.6641 

Parameter 

Std.  dev . s of 

est i nates 

est i ma  te  s 

-0  437 

0.517 

-0  .299 

0 . 332 

-0.169 

0 . 197 

1 .047 

0 . 938 

0 .963 

0 . 877 

0.8  18 

0 . 739 

Stages  run 

900 

Error  3 

2 .6765 

Std. 

dev. 

of  error3 

3 .0766 

Parameter 

Std 

d ev  . s of 

est i nates 

est i nates 

-0.404 

1.213 

-0.263 

0 . 808 

-0.148 

0 . 434 

Hean=  0.00 
Mean®  0.00 


St  d . de  v . = 1.00 
St  d . de  v . 3 1 0 00 


si  3 1.00 
dl  * 0 . 1 Oe  01 


s 2 
d 2 


0 00 

0 . 17e  38 


ucutof  = 0.17e  38 


v 


1 


Jim  7 2 0-' 

4 4 no 

• s e 7 Page 

2 

0.981 

3 965 

0 9 04 

3 69  2 

0 7 63 

3 16  1 

Stages  run 

1200 

Error  = 

6 064  0 

Std 

dev. 

of  error* 

1 4 3386 

Parameter 

Std 

d e v s of 

est  i nates 

est i nates 

0 6 86 

5.761 

0.419 

3.791 

0 199 

2 063 

-2.435 

15 . 940 

-2.279 

1 4 86  6 

-1  9 62 

12  759 

Stages  run 

1500 

Error  = 

2.1818 

Std 

dev. 

of  error* 

3.1740 

Parameter 

Std 

dev ,s  of 

est i nates 

est i nates 

-0.370 

1.043 

-0.231 

0.608 

-0.127 

0 . 297 

1 0 03 

3.744 

0 924 

3.432 

0 . 7S1 

2.981 

Stages  run 

1800 

E r ro  r = 

1.8310 

Std 

dev. 

of  error* 

1.1242 

Para  r i ft s r 
est i Nates 
**0  2 6 8 
-0.182 
-0.1 06 
0 6 06 
0 .638 
0 5 36 


Std  dev  .s  of 
est i nates 
0.762 
0.519 
0.301 
1888 
1.762 
1 . 520 


Stages  run  2100  Error  = 22285 

Std.  dev  of  error:  2.2817 


Paraneter 
est  i it atss 
-0.309 
- 0 2 09 
-0.119 
0 8 99 
0 8 27 
0 8 96 


Std  dev. s of 
est  i nates 
0.993 
0 645 
0.345 
2 997 
2 . 793 
2 397 


* 
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Stages  run 

2400 

Error  «= 

1 

Std  . 

dev. 

of  error* 

0 

Parameter 

Std 

dev .s  of 

estimates 

est i ma  te  s 

-0  332 

0 . 397 

-0.219 

0 . 273 

-0.122 

0.181 

0.971 

0.967 

0 893 

0 . 903 

0 .753 

0 . 778 

Stages  run 

2700 

Error  = 

2 

Std  . 

dev. 

of  errors 

2 

Parameter 

Std 

dev .s  of 

est i mates 

estimates 

-0.527 

1 . 047 

-0  341 

0 . 707 

-0.187 

0 . 404 

1 .8  00 

2.665 

1.480 

2 . 488 

1 ..-'56 

2.139 

Stages  run 

30  0 0- 

Error  = 

1 

Std 

dev. 

of  errors 

1 

Parameter 

Std 

d ev . s of 

est i mates 

estimates 

-0.338 

0 . 580 

-0  224 

0.386 

-0.126 

0 . 220 

0 874 

1 . 485 

0 805 

1 . 386 

0.879 

1 192 
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System  order  n = 3 


The  system  parameters  a r e < i n the  ordsra  1<  . . i an<b 1 < . . < bn  ) s 
-1.500 
0 7 05 

-a  loo 

0 0 65 
0 0 43 
-0  008 

Nominal  noise-'  Gaussian 

Contaminating  noise'-  Laplacian 
Contamination  level  = 0 05 


Hean=  0.00  Std.  dev  = 1 00 

PI  e a n = 0.00  Std  dev  =10  . 00 


The  Hampel  parameters  are'- 


si  - 1.00  s 2 = 0.00 

dl  = 0 . 7 Oe  00  d2  - 0.17e  38 

ucutof  = 0 1 7 e 38 


Stages  run 

300  Error  = 

Std. 

dev.  of  error= 

Parameter 

Std  dev.s  of 

e s t i nates 

est i nates 

-0.325 

0 457 

- 0 2 33 

0 309 

-0  152 

0.192 

0 4 31 

1.491 

0 4 43 

1.390 

0 3 ’9 

1 184 

Stages  r u r. 

600  Error  = 

Std  . 

dev.  of 

error= 

Parameter 

Std  dev.s  of 

est i mates 

e st  i 

mates 

-0  3 3? 

0 . 

555 

-0.2  76 

0 . 

378 

-0  171 

0 . 

23  1 

0.724 

1 . 

652 

0 6 67 

1 

54  2 

0 5 65 

1 

325 

rn 

.4 

to 

e s run 

900 

Error  = 

Std. 

dev. 

of  error® 

P ar  a 

n stir 

Std 

dev.s  of 

e s t i 

n a t e s 

est i mates 

- 1 

0 l 4 

3.743 

-0 

621 

2 21  2 

-0 

3 06 

1.019 

1 6394 
0 6627 


1 682  9 
1 0272 


4 0576 
13  7 168 
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3 

571 

14  327 

3 

3 13 

13.325 

2 

8 22 

11.389 

Stages  run 

1200 

Error  = 1 

Std  . 

dev. 

of  error®  1 

Paraneter 

Std 

dev . s of 

est i nates 

est  i nates 

-0  3 13 

0 . 734 

-0  2 11 

0.471 

-0.123 

0 260 

0 742 

2.230 

0 6 S 1 

2 079 

0 574 

1.785 

Stages  run 

1500 

Error  = 

8 

Std  . 

dev 

of  error® 

35 

Parameter 

Std.  dev .s  of 

est i nates 

est  > nates 

-1.501 

7.762 

-0.782 

3.991 

-0.262 

1.321 

7.118 

37 . 233 

6 600 

34.581 

5 625 

29.539 

Stages  run 

1800 

E r ro  r = 

1 

Std 

dev. 

of  error® 

1 

Paraneter 

Std 

d ev . s of 

est i nates 

e 

st i na  te  s 

-0.471 

0.557 

-0  3 10 

0 . 365 

-0  174 

0.207 

1 .282 

1.584 

1 1 85 

1 . 476 

1.005 

1.267 

Stages  u n 

2100 

Error  = 

7 

Std 

dev. 

of  error® 

28 

Paraneter 

Std 

dev .s  of 

est  > 

nates 

e st i na  te  s 

i 

. 005 

8.143 

0 

652 

5.301 

0 

4 10 

3 165 

-3 

926 

29.548 

-3 

663 

27.500 

_ 3 

1 43 

23.517 

.9298 

.5294 


5212 

.3760 


.6960 

.3070 


.2  183 
1430 


184 
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Stag 

;s  run  2400 

Error  = 

1 8296 

S t d dev. 

of  error: 

1 4 90  7 

Paraneter  S t d 

d ev  . s of 

est  i 

notes  est 

i nates 

-0 

4 06 

0 

663 

-0 

273 

0 

42  1 

-0 

1 55 

0 

227 

1 

042 

2 

00  7 

0 

961 

1 

870 

0 

8 13 

l 

607 

Stages  run 

2 7 0 0 E 

rror  = 

5 7953 

Std 

dev.  of  error= 

205034 

Parameter 

Std 

dev . s of 

est i nates 

est 

i nates 

- 1 674 

6 

. 77  1 

-1.110 

4 

436 

-0  618 

£ 

417 

*.£> 

I-J 

CO 

21 

14  1 

4 5 82 

19 

693 

3 920 

15 

. 89  1 

Stages  run 

3000  Error  = 

2.4496 

Std 

dev  of  error5 

2.6744 

Para  ne  ter 

Std  . 

dev.s  of 

est i nates 

est 

i nates 

1 

O 

CO 

1 

03  1 

-0  119 

0 

66  1 

- 0 0 66 

0 

. 355 

0 3 70 

7 

•J 

53  0 

0 3 35 

3 

332 

0 2 76 

2 

. S52 

185 
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System  order  n = 


The  s y s t e n parameters  are(in  the  ordera  1/  . / an  , b 1 j . . / bn  ) 

-1  5 00 
0 7 05 
-0  100 
0 0 05 
0 0-8 
-0  0 03 


Non i na  ! noise! 

Gauss 

i a n 

M ea  n = 

0 . 00 

Std 

dev 

= 1 

00 

C o n fc  a f.  ' n a t • n g no  ise  : 

Lap  lac 

i a n 

M ea  n = 

0 . 00 

St  d 

dev 

= 10 

. 00 

Continuation  level  = 

0 

05 

The  H a f>  p e l parameters 

are! 

si  = 1.00 
d 1 = 0 1 2e  0 1 

uc  u to  f = 

s 2 
d 2 

0 . 79e  01 

0 o 

1 

II  II 

27 
35  e 

0 1 

Stages  run 

300  Error  = 

1 .5954 

Std . 

dev  of 

error= 

0.9244 

Parameter 

Std  dev.s  of 

e s t i mates 

? at  1 

nates 

- 0 2 v 4 

0 

534 

- 0 10  3 

0 . 

370 

- 0 1 1 3 

0 

223 

0 4 07 

1 . 

563 

0 - 5 7 

1 . 

455 

0.381 

1 

242 

3 

r u r 

600  £ 

r r o r = 

1 6 

8623 

Std  . 

dev.  o 

f error5 

80 

.7389 

P 

4 r & ft  e 

ter 

Std  . 

dev.s  of 

p 

SC  t ft  4 

t e s 

est 

! nates 

“ **  o 

<?  c 

j si 

n 4 
&.  L 

.188 

_ e 

c.  -J 

vi  e. 

12 

. 559 

- i 2 

?b 

6 

225 

1 O f 

03 

83 

587 

1 5 5 

27 

-»  •» 

1 t 

739 

13  2 

o 1 

6 fe 

. 429 

Stages  ran 

900  Error  = 

1 5590 

Std. 

dev  of  error5 

1 2212 

Para m e t e r 

Std  dev.s  of 

est i nates 

est i nates 

_ cj  j c 7 

0 700 

- 0 133 

0 463 

-0  0 33 

0 26  4 

136 
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Stager  r v ri 

24  00  E 

rror  = 

1 9350 

Std 

dev.  of  error3 

1 .5756 

P a r a r 

i?ter 

Std 

dev .s  of 

est  i r 

. a t e s 

e s t 

i nates 

* 0 

1 93 

0 

772 

- 0 

: 23 

0 

.518 

- 0 

0 v2 

0 

297 

0 

5 33 

c 

302 

0 

4 S 6 

c! 

.145 

G 

4 04 

i 

839 

S * u ' j r i rufi 

2700  Error  = 

1 4697 

Std 

dev  o 

f e r r o r = 

0.5250 

Para  r-. eter 

Std  . 

dev .s  of 

e s * s r.  u*.es 

est 

i nates 

“ 0 cot 

0 

379 

- 0 1 v 7 

0 

25  1 

-0  10? 

0 

144 

0 S 2 4 

1 

04  9 

0 7 * 7 

0 

97  8 

0 ©77 

0 

839 

S v Ci  9 ^ - run 

3000  Er 

ror  = 

2 

.1726 

S t d 

dev.  of 

error® 

1 

.6947 

Para  r . e t : e r 

Std.  dev  ,s  of 

est i hates 

est  i 

nates 

- 0 7 0 1 

0 

903 

“0  .201 

0 

612 

- 0 1 1 2 

0 

357 

0 "01 

2 

530 

o 7 : s 

£ 

359 

0 6 04 

t 

026 

188 
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System  order  n = 3 


The  3 y s t e n 
-1  5 00 
0 7 0 5 
-0 . i 00 
0 0 35 

0 0 •*  s 

-0  GOO 


parameters 


are*  in 


the  order  a 1 


an  , b 1 


b n ) 


Mot 

. » n<i 

' r>  o i 

lie*. 

Gauss) an 

11  e a n = 0.00 

St  d 

dev  = 1 

. 0 'I 

C or 

1 x,  '*  ?i 

. n «i  t 

i r, g no 

: Jfl  : 

L a p 1 a c i an 

11  e a n = 0.00 

St  d 

dev  =10 

. 00 

C o r.  t a : > 

• n t 

on  l e 

vel  = 

0 05 

T b e H a i 

,pe  ! 

p a r <i  m 

eters 

are: 

si  = 1.00 
dl  = 0 . 25e  01 
ucutof  = 

s 2 
d 2 

0 . 9 5 e 01 

= -0  50 
= 0 45  e 

0 1 

Stages  run  3 G C 
Std.  d e v 


Error  = 2.0325 

of  error=  2.4504 


Para: 

■ieter 

Std  d e v . s 

e s t i r 

rites 

estimate 

-0 

0 1 9 

1.024 

-0 

.131 

0.517 

-0 

£24 

0 *37  2 

0 

042 

2 915 

-0 

055 

2.933 

-0 

0 03 

2.415 

Stages  r u n 

3 0 0 E 

*5 

"5 

O 

T 

II 

1 S57G 

Std  . 

dev.  c 

f error: 

1 .7332 

Parameter 

Std 

dev  s of 

e s t i nates 

est 

n a t e s 

- 0 2^4 

0 

343 

-0  2 02 

0 

403 

-0  1 23 

0 

227 

0 5 32 

2 

39  9 

0 a 90 

O 

c. 

223 

0 a l a 

39  9 

Stag 

e £ r 

u n 

90  0 E 

r 

r o r = 

5 

5 7 9 3 

td 

dev  o 

f 

e r r o r = 

1 O 

9777 

P ar  a 

r. 

r 

Std 

N 

9 V C of 

o z t i 

rl  u 

£ 

e j t 

i 

nates 

- 0 

7 *:© 

3 

98  6 

-0 

V . f 

99  4 

- 0 


503 


0 


189 


0 0 0 4 


no. Page 


r 


3 ^50 

18 

285 

3 1 9 0 

1 6 . 

9 6 8 

2 7 0? 

! 4 

463 

Stases  run 

12  0 0 E r 

r o r = 

2 

6864 

S t d 

d e v c f 

errcr= 

c 

O 

4 072 

P <i r cheter 

S t d aev  s of 

esc  t h a z e s 

e s t i 

hates 

- 0 0 6 8 

2 

15  1 

-0.025 

430 

“0  0 It  £ 

u . 

74  ? 

0 * £ 1 

6 

0 0 i 

» j ^ 3c 

C- 

594 

0 3 6 0 

4 

797 

Stages 

r u n 1 

500  Error  = 

1 .9143 

S t d 

cl  e v of  error1 

1 .1135 

P uf  Uii? 

r.  e r 

Std  dews  of 

e s t ; r.u 

c e s 

est i nates 

- 0 2 

0 7 

0 67  5 

- 0 1 

3 2 

0 4 6 5 

“0.0 

05 

0 24  8 

0 . 5 

3 4 

1 98  6 

0 “ 

vC 

1 846 

u 4 

L« . 

1 584 

Siege 

s r u n 

1 8 0 0 E r 

r o r = 

2 .0  837 

St  c . 

dev.  of 

err  o r = 

2.8338 

Par  a i . 

e ter 

Std  d 

e v . s of 

est:  h 

cces 

est  i 

hates 

“0 

r * o 

* Ci 

i . 

439 

- 0 

3 3 3 

0 . 

9 1 1 

- G 

1 0 3 

G 

54  7 

1 

3:8 

7 

1 ei  2 

n • •* 
. i 

t . 

93  1 

i 

•J  3 O 

52  9 

r 


ur, 


12  0 0 0 4 


no  i;ef 


Page  3 


S t ..  g * s r u n 

2 4 0 0 E r r o r = 

S t d 

d e oT  error* 

Parameter 

3 1 cl  . dev  5 of 

9 3“  1 Pt4  ^93 

est  i hates 

- 0 5 31 

1 102 

- 0 3 3 7 

0.730 

-a  2i7 

0 401 

: 5 

3 194 

1 .4  30 

2 97  3 

1 5 7 

2 555 

Stages  run 

2700  Error  = 

3 * d . 

dev  of  error* 

Para  i.  eter 

Std  dev  s of 

est i nates 

est  i nates 

-0  2 33 

G . S59 

-0.1 32 

0 577 

- 0 101 

0 32  7 

0 6 26 

2 563 

0 6 29 

2 393 

0 5 2 8 

2 05  1 

Stag 

9*  run 

3000 

Error  = 

Std. 

dev. 

0 

f error* 

Para 

n eter 

Std 

dev .s  of 

esti 

Rates 

est 

i nates 

- 0 

.453 

i 

84  6 

-0 

2 “*2 

i 

. 223 

-0 

• 28 

0 

715 

i 

84? 

4 

750 

1 

705 

4 

43  1 

4 

442 

7 

.311 

:.’38? 
2 0 7 5 4 


2.0355 
1 350  3 


3 0 649 
4 .0915 
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System  order  n = 2 


The  system  parameters  a r e ( i n the  orderali  . . < an  .b  1 i . . i bn  ) : 

-1.423 
0.49b 
0 0 00 
-0.1 02 
0.173 
0 .0  00 


nominal  noise: 

G a us  s i an 

M e a n = 

0 . 00 

St  d 

dev 

. = 1 

. 00 

Contaminating  noise: 

La  p l a c i a n 

PI  ea  n = 

0 . 00 

Std 

dev 

= 10 

. 00 

Contamination  level  = 

0.03 

The  Han pel  parameters 

are: 

si  = 1.00 
dl  = 0.17e  38 

uc  u to  f = 0 . 

s 2 
d 2 

1 7 e 38 

= 1 . 

= 0 . 

00 
1 7 e 

38 

Stages  run 

300 

E r r o r = 

Std  . 

dev. 

of  error= 

Parameter 

Std 

dev ,s  of 

est i mates 

est i mates 

-0.227 

2 . 600 

1.046 

2.714 

-0.061 

0.247 

0.216 

0.521 

Stages  ron 
Std. 

600 

dev. 

Er ror  = 

of  error: 

Parameter 
es’c  i mates 
0 4 63 
1.420 
- 0 .067 
0.064 

■Std  dev. s of 
est i nates 
2.043 
5.121 
0.187 
0.442 

Stages  run 
Std. 

900 

dev. 

E r r o r = 

of  error= 

Parameter 
est i nates 
-0  3 92 
1 111 
-0.0  64 
0.116 

Std.  dev.s  of 
est i mates 
1 228 
2 244 

0.131 
0 . 270 

Stages  run 

1200 

E r r o r = 

2.0169 
1 .7098 


2 2687 
3.1596 


1 .4519 
1 1 753 


1 9809 


192 


"H 
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I 


Std . 

dev 

of  error= 

3 357  1 

Par  a he  ter 

Std 

d e v s of 

e s t i hates 

0 552 

1 i 87 
-0.101 
-0.05b 

est i nates 
5.019 
2.221 
0.155 
0 . 787 

Stages  run 

1500 

Error  = 

1 .0842 

Std. 

dev. 

of  error® 

0.5394 

Parameter 

Std 

dev . s of 

est i nates 
-0.472 
0 723 
-0  OS? 
0.102 

e 

st  mates 
1.001 
1.177 
0 . 105 
0.188 

Stages  run 
Std. 

1800 

dev. 

Error  = 

of  error® 

1 .3335 
1.1961 

Parameter 
est i nates 
-0.181 
0 9 66 
-0 . 0S3 
0.020 

Std.  dev.s  of 
estimates 
1.762 
1.515 
0.109 
0 . 422 

Stages  run 
Std. 

2100 

dev. 

Error  = 

of  error® 

1 .3504 
0 .9448 

Parameter 
est i nates 
-0  2 02 
1.093 
-0.092 
0.049 

Std.  dev.s  of 
est  i nates 
1 147 

1.746 
0 . 097 
0.228 

Stages  run 

2400 

E r r o r = 

1 5304 

Std. 

dev. 

of  error® 

1.1430 

Parameter 

Std 

dev.s  of 

est i nates 

est i nates 

-0.101 

1 908 

0 3 68 

1 665 

-0.068 

0.201 

0.073 

0.469 

Stages  run  2700 

Error  = 

1.2194 

Std.  dev 

of  error® 

1 1478 

4 * 


1 


J uo  9 09>56 


Paranettr 
est  i incites 
-0  . £'3 
0 395 
-0  080 
0.033 


Para  Meter 
est  i mates 
-0  724 
0 342 
-0.072 
0.074 
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Std  dev.s  of 
estimates 
1.761 
1 328 
0.123 
0.509 


E r r o r = 

of  e rr o r = 

Std  dev.s  of 
estimates 
1.408 
1 . 208 
0 178 
0 . 350 


Stages  run  3000 
Std.  dev. 


1 0457 
0 .8278 
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System  order  n 3 2 


The  system  parameters  are(  in  the  orderal..  / an  * b 1 / . . bn  ) : 

-1.425 
0 4 98 
0 0 00 
-0  102 
0 173 
0 000 

Nominal  noise:  Gaussian  Nean=  0.00  Std.  dev  .=  1.00 

Contaminating  noise!  Laplacian  flean»  0.00  Std.  dew.  = 10. 00 

Contamination  level  3 0 03 

The  Hampel  parameters  are:  si  3 1.00  s2  3 0 00 

dl  3 0 . 2 Oe  0 1 d2  = 0 . 1 7 e 38 

uc  u to  f = 0.17e  38 


Stages  run 

300 

Error  3 

1.4660 

Std  . 

dew. 

of  error= 

1.2774 

Parameter 

Std 

dews  of 

e s t i mates 

est i nates 

0 0 10 

2 . 054 

0 279 

1 . 529 

-0.114 

0 . 162 

0 0 80 

0.244 

Stages  run 

600 

E r r o r 3 

2.0116 

Std  . 

dev. 

of  error3 

2 .5007 

Parameter 

Std 

dews  of 

est i mates 
- 0 253 
0 338 
-0.115 
0 151 

est i nates 
2.737 
3 . 897 
0.131 
0 . 236 

Stages  run 

900 

Error  3 

0.9692 

Std. 

dew. 

of  error3 

0.7315 

Paraneter 

Std 

dews  of 

est i nates 

est i nates 

-0  3 79 

0 343 

1 015 

1.451 

-0  0 96 

0.113 

0 101 

0 129 

Stages  run 

1200 

E r r o r 3 

0 9284 

195 
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Std.  dev.  of  error1 


Par  oheter 
est i notes 
-0 .778 
1 G 05 
-0.119 
0 077 


Std  dev .s  of 
est i mates 
0.953 
1.388 
0 102 
0.178 


Stages  run  1500  Error  = 
Std.  dev  of  errors 


Parameter 
est i mates 
- 1 . 0S3 
0.808 
-0  123 
0.121 


Std  dev .s  of 
est i nates 
1.396 
0.878 
0.098 
0 174 


Stages  run  1800  Error  = 
Std.  dev.  of  error= 


Parameter 
est i nates 
-0.632 
0.848 
-0.102 
0.092 


Std  dev . s of 
est i nates 
0.876 
0.707 
0.058 
0.133 


Stages  run  2100  Error  = 
Std  dev.  of  error= 


Parameter 
est i nates 
- 1 .147 
0 789 
-0.099 
0 127 


Std.  dev .s  of 
est i nates 
0.812 
0.880 
0.067 
0.118 


Stages  run  2400  Error  = 
Std.  dev.  of  errors 


Parameter 
estimates 
- 1 193 
0 7 05 
-0.106 
0 1 34 


Std.  dev  .s  of 
est i nates 
0.582 
0 894 
0 050 
0.121 


Stages  run  2700 
Std  dev 


Error  = 
of  error* 


0 8229 


0 . 864e 
0 .7324 


0 . 7208 
0.4948 


0 .7009 
0.4543 


0.6421 

0.3588 


0.5402 

0.3332 


196 
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System  order  n = 2 


The  system  parameters  are<  in  the  orderal<  . . ;an<bl<  . ,<bn)  : 

-1.425 
0 .496 
0.000 
-0.102 
0.173 
0 .000 

Nominal  noise:  Gaussian  dean3  0.00  Std.  dev.=  1.00 

Contaminating  noise:  Laplacian  dean3  0.00  Std.  dev. =10. 00 

Contamination  level  3 0.03 

The  Hampel  parameters  are:  si  3 1.00  s2  3 0.00 

dl  3 0.1 5e  01  d2  3 0 . 1 7 e 38 

ucutof  3 0 . 1 7e  38 


Stages  run  300  Error  3 1.3457 

Std.  dev.  of  error3  1.0236 

Parameter  Std.  dev  .s  of 
estimates  estimates 

-0084  1.489 

0 289  1.577 

-0.107  0.116 

0 049  0.222 


Stages  run  600  Error  3 1.4969 

Std.  dev.  of  error3  1.8338 

Parameter  Std.  dev  s of 
estimates  estimates 

-0  295  1.917 

0.578  2.821 

-0.095  0.139 

0.137  0.219 


Stages  run  900  Error  3 0.8086 

Std  dev.  of  error*  0 2996 

Parameter  Std  dev  s of 
estimates  estimates 

-0  806  0.676 

0820  0 863 

. -0.089  0.116 

0.106  0.125 

f Stages  run  1200  Error  3 0.7924 

198 
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Std  dev.  of  error®  0.6257 


Par  a me te  r 
estimates 
-0.894 
0 .935 
-0.103 
0.099 


Std  dev  . s of 
estimates 

0 . 874 

1 044 
0.081 
0 . 147 


Stages  run  1500  Error  = 06885 

Std.  dev.  of  error*  0.3483 


Parameter 
est  i nates 
-1.009 
0 6 98 
-0.098 
0.128 


Std.  dev  s of 
est i nates 
0 . 724 
0.786 
0 . 069 
0 . 129 


Stages  run  1800  Error  = 0.6322 

Std.  dev.  of  error®  0.3401 


Paranster 
estimates 
-0  .977 
0.802 
-0.115 
0 .096 


Std.  d ev  . s of 
estimates 
0 . 620 
0 . 689 
0 . 107 
0 . 153 


Stages  run  2100  Error  = 0.6429 

Std.  dev.  of  error*  0.3655 


Parameter 

estimates 

-1.117 

0.715 

-0.102 

0.124 


Std  . dev . s of 
est  i nates 
0.710 
0 . 777 
0 . 068 
0 . 104 


Stages  run  2400  Error  = 0.5951 

Std  dev.  of  error*  0.2805 


Parameter 
est  ; mates 
-1.195 
0.650 
-0.104 
0.137 


Std.  d ev  . s of 
est i nates 
0.571 
0.764 
0 . 056 
0.113 


Stages  run  2700  Error  ■ 06094 

Std.  dev.  of  errors  05410 


- ,*.v» 


v ur 


? 1 C 


i 


noise?  Page  3 


P a r a r.  e t e r Std.  dev  s of 

estiKctes  est i nates 

- 1 390  102  0 

0 505  0688 

-0  108  0.071 

0 14?  0 099 


Stages  run  3000  Error  = 0.4605 

Std  dev  of  error*  0.2328 

Parameter  Std  dev.s  of 
estimates  est i nates 

-1  18?  0.483 

0542  0 . 558 

-0  G?6  0 . 060 

0.139  0.099 
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System  order  n 3 2 


The  system  parameters  are<  in  the  ordoral  i . . < an( bl  / . . i bn)  : 
-1.425 
0.496 
0.000 
-0.102 
0.173 
0 .000 


Nominal  noise:  Gaussian 

Contaminating  noise:  Laplacian 
Contamination  level  3 0.03 

The  Hampel  parameters  are: 


Mean3  0.00  Std.  dev.3  1.00 

Mean3  0.00  Std.  dev. >10. 00 


si  3 1 . 00  s230.0Q 

dl  3 0. lOe  01  d2  3 0 . 17e  38 

ucutof  3 0 . 1 7e  38 


Stages  run 

300 

Error  3 

1 .4043 

Std. 

de  v . 

of  error* 

1 .2021 

Parameter 

Std 

dev .s  of 

estimates 

est i mates 

-0.122 

l . 474 

0 .473 

1 . 986 

-0.100 

0 . 130 

0.041 

0 239 

Stages  run 

600 

Error  3 

1.3136 

Std. 

dev. 

of  error3 

1 .6293 

Parameter 

Std 

. d ev  . s of 

estimates 

est i nates 

-0.417 

1 . 680 

0.185 

2 . 477 

-0.078 

0 . 172 

0 148 

0.218 

Stages  run 

900 

Error  = 

0 .7768 

Std . 

dev. 

of  error3 

0 3861 

P ar  a me  ter 

Std 

d ev . s of 

est i nates 
-0  898 
0 8 05 
-0  088 
0.126 

estimates 
0 726 
0 905 
0 109 
0 100 

Stages  run  1200  Error 


0 .6979 


201 
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Std.  dev.  of  error*  0.3140 


Paraneter 
estimates 
- 1 .061 
0.881 
-0  116 
0.108 


Std . dev . s of 
est  i nates 
0.711 
0 . 962 
0 . 078 
0.141 


Stages  run  1500  Error  = 
Std.  dev.  of  error* 


0 .6606 
0 .3223 


P ar  a ne  te  r 
est i nates 
- 1 .156 
0 .652 
-0.119 
0.134 


Std . dev . s of 
est i nates 
0 . 606 
0 . 874 
0.067 
0 . 133 


Stages  run  1800  Error  = 
Std.  dev.  of  error* 


0 .5623 
0.3174 


P ar  a ne ter 
est  i lutes 
-1  .068 
0 .748 
-G .101 
0.124 


Std  dev  . s of 
est i nates 
0 . 487 
0 . 724 
0 . 058 
0.087 


Stages  run  2100  Error  = 
Std.  dev.  of  error* 


0 .5848 
0 .3730 


Parameter 
est i nates 
-1.115 
0 665 
-0.100 
0.133 


Std  d ev . s of 
e st  i na  te s 
0 . 602 
0 . 78  3 
0 . 059 
0 . 100 


Stages  run  2400  Error  * 
Std.  dev.  of  error* 


0 3704 
0.2803 


Para  he  ter 
est i nates 
-1.221 
0 626 
-0  100 
0 147 


Std.  d ev  . s of 
e st i na  te  s 
0 608 
0.701 
0 057 
0 106 


Stages  run  2700  Error  * 0.4983 

Std  dev  of  error*  02414 

202 
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Parameter 

Std  dev  . s of 

est i nates 

est i nates 

-1.282 

0 . 550 

0.516 

0.612 

-0  105 

0 . 058 

0.144 

0 . 095 

Stages  run 

3000 

Error  = 

0.4482 

Std  . 

dev. 

of  error* 

0 .203  1 

Paraneter 

Std 

dev . s of 

est i nates 

e st  i na te s 

-1  223  . 

0 . 488 

0 550 

0.516 

-0.101 

0 . 060 

0.140 

0 . 086 
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Systen  order  n = 2 


The  systen  paraneters  »re(  in  the  order  a 1 , . . / an  / b 1 . . . < bn  ) : 
-1.423 
0 .496 
0.000 
-0.102 
0 173 
0 .000 

Noninal  noise:  Gaussian 

Contan i nat i ng  noise:  Laplacian 
C on t an i n a t i on  level  = 0.03 

The  Hanpel  paraneters  are: 


Mean®  0.00 
Mean®  0.00 


St  d . dev.®  1.00 
Std.  dev. =10. 00 


si  = 1 . 00  s2  = 0 . 00 

dl  = 0 . 7 Oe  00  d 2 = Q.17e  38 

ucutof  = 0 . 1 7e  38 


Stages  run 

300 

Error  » 

1 .6069 

Std. 

dev . 

of  error® 

l .7903 

Par aneter 

Std 

. dev . s of 

est i na  tes 
-0  .456 
-0 . 101 
-0.100 
-0  009 

est i nates 

1 . 725 

2 997 
0 . 132 
0 . 327 

Stages  run  600  Error  * 
Std . dev . of  error® 


1 .4834 
1 .6090 


Par aneter 
est i nates 
-0 .484 
0 303 
-0  092 
0 128 


Std.  dev  . s of 
est i nates 

1 963 

2 . 509 
0.151 
0.214 


Stages  run 

900 

Error  * 

1.1430 

Std 

de  v . 

of  error® 

1.6198 

Par  ane  ter 

Std 

dev . s of 

e3tinates 
-1.046 
1.307 
-0  141 
0 096 

est i nates 
0 . 979 
2 693 
0 199 
0.213 

Stages  run  1200  Error 


0 7649 
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Std.  dev.  of  error*  05410 


Paraneter 
e st i nates 
-0  9 75 
0 .9  04 
-0  104 
0 1 16 


Std.  dev  . s of 
est  i nates 

0 . 739 

1 . 045 
0 . 074 
0 . 133 


Stages  run  1500  Error  * 0.7472 

Std.  dev.  of  error*  0.3359 


Par  a me  ter 
est i nates 
-1167 
0.669 
-0.118 
0 1 44 


Std.  d ev  . s of 
est i nates 
0 . 695 
0 . 967 
0.110 
0 . 163 


Stages  run  1800  Error  = 06676 

Std.  dev.  of  error*  0.3198 


Paraneter 
est i nates 
-1.081 
0 . 786 
-0.104 
0.129 


Std.  d ev  . s of 
est  i nates 
0.571 
0.848 
0.081 
0.114 


Stages  run  2100  Error  * 0.6704 

Std  dev.  of  error*  04877 


Paraneter 
est  i nates 
-1.168 
0 620 
-0  107 
0 140 


Std.  d ev  . s of 
est i nates 
0 . 796 
0 . 925 
0 . 066 
0 . 129 


Stages  run  2400  Error  * 
Std.  dev.  of  error* 


Par aneter 

est  r nates 
-1.184 
0 6 11 
-0  103 
0 1 44 


Std.  dev  . s of 
est i nates 
0 604 
0 745 
0 . 056 
0 116 


Stages  run  2700  Error  « 
Std.  dev.  of  error1 


*1W*1  s.$ii  n»*>* 


0 6177 
0 .2320 


0 5318 
0 2000 


Jur.  9 10:40  noise?  Page  3 


Para  he  ter 

S td  dev  s of 

e st i nates 

est i nates 

-1  2 15 

0.514 

0 530 

0 652 

-0  096 

0 059 

0.145 

0 097 

Stages  run 

3000 

E r ro  r = 

S t d . 

dev. 

of  error® 

P ar  a ne  te  r 

S td 

dev .s  of 

est i nates 

est i nates 

- 1 .235 

0 . 499 

0.580 

0 . 558 

-0.100 

0 . 055 

0.141 

0 . 087 

0 .4738 
0 .2026 


i 
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System  order  n = 2 


The  systen  parameters  are(  in  the  ordera  1 1 . . / an / b 1 / . . , bn)  : 

-1.425 
0 4 96 
0 .0  00 
-0.102 
0 1 73 
0 .0  00 

Nominal  n o i s e : Gaussian  Mean®  0.00  Std.  dev  .=  100 

Contaminating  noise:  Laplacian  Mean*  0.00  Std.  dev.  = 10  00 

Contanination  level  = 0.03 


The  Hampel  parameters  are: 


si  * 1 . 00  s2  =-0 . 2? 

dl  * Q 1 2e  01  d2  * 0 . 35  e 01 

ucutof  = 0 . 7 9 e 01 


Stages  run 

300 

E r r o r = 

1 .2634 

Std  . 

dev 

of  error® 

0.9080 

Parameter 

Std 

d ev  s of 

e s t i mates 
-0.231 
0.406 
-0  0 90 
0 .051 

e st i mates 
1 366 
1.489 
0 . 132 
0 . 242 

Stages  run 

600 

Error  = 

1.1772 

Std  . 

dev. 

of  error® 

0 .9690 

Parameter 

Std 

d ev  . s of 

est i mates 

estimates 

- 0 4 75 

1.331 

0 3 93 

1611 

-0  0^5 

0.210 

0.130 

0.244 

Stages  run 

900 

Error  = 

0.7411 

Std  . 

dev  . 

of  error® 

0.3114 

Parameter 

Std 

d ev  s of 

est i mates 
-0  869 
0 742 
-0  109 
0 1 06 

est i mates 
0 . 628 
0 . 838 
0 . 098 
0 133 

Stages  run 

1200 

E r r o r ® 

0.6471 
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Std  . 

dev 

of  error* 

0.5154 

Parameter 
est inotes 
- 1 . 1 £1 
0 8 36 
-0  113 
0 123 

Std 

e 

d ev . s of 
st  i na  te  s 
0 644 
0 . 983 
0 096 
0.134 

Stages  run 
Std  . 

1500 

dev. 

Error  = 

of  error* 

0 6653 
0 3295 

Parameter 
est i mates 
-1.186 
0 6 57 
-0.114 
0.144 

Std  dev  .s  of 
est i nates 
0.679 
0.843 
0.063 
0.128 

Stages  run 

1800  Error  * 

0 6234 

Std 

dev.  of 

errors 

0.3359 

Paraneter 

Std.  devs  of 

est i mates 

e s t i 

nates 

-1  189 

0 

649 

0.792 

0 . 

764 

-0.101 

0 

066 

0 1 37 

0 . 

08  3 

Stages  run 

2100 

Error  = 

0 .5483 

Std 

dev. 

of  error* 

0 .365  1 

Parameter 

Std 

devs  of 

est i nates 

est i nates 

-1.112 

0 558 

0 . 6’2 

0 743 

-0.101 

0 053 

0 126 

0 . 090 

Stages  run 

2400 

Error  = 

0.5360 

Std. 

dev. 

of  error* 

0.2712 

Parameter 

Std 

devs  of 

est i nates 

e 

st i nates 

-1.202 

0.546 

0 622 

0 677 

-0  105 

0.057 

0.138 

0 . 096 

Stages  run  2700  Error  = 
Std.  ciev.  of  error* 


0 4694 
0 .2425 
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0.145  0 085 


Stages  run  3000  Error  = 04612 

S t d . dev.  of  error®  02068 

Parameter  Std.  dev.s  of 
esti nates  estimates 

-1  262  0.500 

0 506  0.538 

-0.114  0.057 

0 1 30  0.102 


w 


.*  • 


J u n 12  09:33  noise?  Page  1 


System  order  n = 2 


The  system  parameters  are(  in  the  orderal  < . < an . b! i . . / bn)  : 

-1  4 25 
0 4 9b 
0 0 00 
-0.1 02 
0 1 73 
0 0 00 

Nominal  noise-  Gaussian  Mean3  0.00  Std 

Contaminating  noise-  Laplacian  Mean3  0.00  Std. 

Contamination  level  = 0.03 

The  Hampel  parameters  are:  si  =1.00  s2 

dl  = 0 2 5e  01  d 2 

ucutof  = 0 . 9 5 e 01 


dev . * 1.00 

de  v . = 1 0 00 

= -0  50 
= 0 45e  01 


Stages  run 
Std  . 


300  Er ror  3 
dev.  of  error3 


1.2220 
0 .8375 


Parameter 

Std  dev .s  of 

e s t i mates 

e st i mates 

-0.284 

1.465 

0.180 

1 . 202 

-0.107 

0 . 122 

0.046 

0.251 

Stages  run 

600 

Error  = 

Std  . 

dev. 

of  error3 

Parameter 

Std 

d e v . s of 

est i mates 

est i nates 

-0.842 

1 . 857 

1 256 

2 588 

• - 0 101 

0.119 

0.096 

0.221 

Stages  run 

900 

Error  3 

Std 

dev 

of 

error2 

Parameter 

Std 

d e v . s of 

est i mates 

e st  i 

ma  te  s 

-0  SOO 

0 . 

615 

0 8 80 

0 . 

933 

-0  0 89 

0 . 

1 1 5 

0 110 

0 . 

132 

Stages  run 

1200 

Error  3 

1 .5326 
1 .5674 


0 8169 
0.3452 


1.1325 
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Std 

dev.  of 

error'  1.4841 

Parana  ter 

S td  d ev  s of 

estimates 

e st  i 

nates 

-0  5 11 

2 . 

187 

0 9 0 a 

i . 

476 

-0  105 

0 . 

075 

0.058 

0 . 

299 

Stages  run 

1500 

Error  = 

0.7914 

Std 

dev. 

of  error' 

0.4674 

Pararieter 

Std 

dev . s of 

est i nates 

e st  i na te  s 

-0 .975 

0.918 

0.778 

0 . 895 

-0  107 

0.074 

0.124 

0.157 

Stages  run 

1300 

Error  = 

0.7409 

Std  . 

dev. 

of  error= 

0.4808 

Parameter 

Std 

dev .s  of 

e st i na  te  s 

est i nates 

-0  9 77 

0 895 

0 921 

0 768 

-0  128 

0 . 075 

0.086 

0.146 

Stages  run 
Std. 

2100 

dev. 

Error  3 

of  error3 

0.6807 

0.4039 

Paraneter 
est i nates 
-1.136 
0.790 
-0.098 
0.123 

Std  dev .s  of 
est i nates 
0 . 829 
0 . 760 
0.061 
0.101 

Stages  run 
Std. 

24  00 
dev  . 

Er  ror  3 

of  error3 

0.5881 
0 2536 

Paraneter 
est i nates 
-1  2 09 
0 6 65 
-0  0 98 
0 1 ■*  3 

Std 

e 

dev . s of 
st i nates 
0.553 
0.740 
0.058 
0 104 

Stages  run  2700 

Error  3 

0 5252 

Std  dev. 

of  error* 

0 2799 
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Parameter 

Std  dev .s  of 

e st  i Mite: 

e st  i nates 

- 1 2 SO 

0 564 

0 57  1 

0 678 

-0  102 

0 . 060 

0 1 45 

0.107 

St toes  run 

3000 

Error  = 

0 .5372 

Std 

dev 

of  error* 

0.2311 

Par  a r< ?ter 

Std 

dev .s  of 

e s t i nates 

e st i nates 

-1  2 49 

0 . 542 

0 . 6 5 C 

0.654 

-0  111 

0.049 

0.134 

0.109 
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System  order  n = 2 


The  system  Darameters  a r e C i n the  orderal,  . . < nn<bl , . . < bn)  : 

-1.425 
0 . 496 
0 .000 
-0.102 
0.173 
0 .000 

Nominal  noise:  Gaussian  Mean*  0.00  Std.  dev*  1.00 

C on t an i n a t i ng  noise:  Gaussian  Mean*  0.00  Std.  dev. *10. 00 

C on t an i n a t i on  level  = 0.10 

The  Hampel  parameters  are:  si  *1.00  s2*1.00 

dl  = 0 . 1 7e  38  d2  * 0 17e  38 

ucutof  = 0 . 1 7e  38 


Stages  run  300  Error  = 1.4392 

Std.  dev.  of  error*  0.8631 

Parameter  Std.  dev.s  of 
estimates  estimates 

-0.125  1.417 

1 083  1.419 

-0.240  0.311 

0.101  0.592 
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Std  dev.  of  error®  0.9163 

Parameter  Std.  dev.s  of 
esti hates  estimates 

-0.225  1.433 

G 856  1 . 704 

-0.111  0.227 

0059  0.344 


Stages  run  1500  Error  = 1.2922 

Std  dev.  of  error®  1.2109 


Parameter 
est i nates 
-0.162 
0.053 
-0.205 
-0  006 


Std  . dev  .s  of 
est inates 
1.325 
l . 663 
0 . 664 
0.687 


Stages  run  1800  Error  = 1.2868 

Std  dev.  of  error®  1.2070 


Paraneter 
est  i nates 
-0  .533 
0 .750 
-0.229 
0.100 


Std.  dev.s  of 
estimates 
1 . 337 
1.858 
0.719 
0 . 758 


Stages  run  2100  Error  = 1.0818 

Std  dev.  of  error®  0.7483 


Parameter 
est  : nates 
-0  434 
0 .762 
-0.129 
0.137 


Std.  dev.s  of 
est  i nates 

0 . 964 

1 . 396 
0 . 148 
0 . 235 


Stages  run  2400  Error  = 1.3361 

Std  dev.  of  error®  1.5847 


Paraneter 
est  i nates 
-0  320 
0 3 09 
-0  105 
0 156 


Std  dev.s  of 
est i na  te  s 

1 349 

2 607 
0.170 
0 233 


Stages  run  2700  Error  * 1 .4829 

Std  dev  of  error®  1.4044 
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Parameter 

Std.  d ev  s of 

est i nates 

est i nates 

-0.780 

2.088 

1.011 

2.115 

-0  116 

0 . 160 

0.182 

0 . 352 

Stages  run 

3000 

Error  “ 

Std  . 

dev . 

of  error* 

Paraneter 

Std 

. dev  s of 

est i nates 

est i nates 

-0.119 

1.571 

0 .061 

2 . 670 

-0 .091 

0.178 

0.144 

0 . 227 

<.>.«  ■ ■ f'-  ^ V"»  .-.  . *»’ , « ■ — ■ 
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System  order  n = 2 


The  system  parameters  are<  in  the  order  a 1 < . . < an < b 1 > . . < bn ) : 
-1.425 
0.496 
0.000 
-0.102 
0.173 
0.000 


Nominal  noise:  Gaussian 

Contaminating  noise:  Gaussian 


C on  t an inati 

i on  level  3 

0 

. 10 

The  Hampel 

parameters  are: 

Stages  run 

300 

Error 

3 

1 .9327 

Std . 

dev. 

of  error® 

3 .2329 

Par ame  ter 

Std 

dev  . s 

of 

e st  i nates 

estimates 

-0  .791 

2 . 406 

-0.408 

5 . 070 

-0.134 

0.189 

0 .030 

0 . 324 

Stages  run 

600 

Error 

3 

1 .6729 

Std . 

dev. 

of  error3 

1.7616 

Parameter 

Std 

dev  . s 

of 

est i mates 

estimates 

-0 .045 

1 . 350 

1 247 

3 031 

-0.103 

0 . 149 

-0 .031 

0 . 363 

Stages  run 

900 

Error 

3 

7.8322 

Std . 

dev 

of  error* 

33 .8222 

Par aneter 

Std 

dev  . s 

of 

estimates 

estimates 

1.407 

9 563 

10.920 

33 . 908 

0 .074 

1 . 026 

0 219 

0 . 870 

Stages  run 

1200 

Error 

3 

12313 

Mean*  0.00 
Mean*  0.00 


si  * 1.00 
dl  * 0 20e  0 1 


Std  . dev  . = 1.00 
Std  . dev  . = 10  00 


s 2 * 0 00 
d 2 * 0 17e  38 


ucutof  * 0 . 1 7e  38 
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Std.  dev.  of  error*  1.0306 


Parameter 
est i nates 
-0.313 
0 434 
-0  116 
0.105 


Std.  d ev  s of 
e st i na  te  s 
1 . 344 
1 866 
0 . 134 
0 . 200 


Stages  run  1500  Error  * 1.1218 

Std.  dev.  of  error*  1.0439 


Paraneter 
est i nates 
-0.823 
0 .687 
-0.113 
0.150 


Std . dev . s of 
est i nates 
1.641 
1.510 
0 . 127 
0 . 225 


Stages  run  180Q  Error  = 
Std.  dev.  of  error* 


0 . 8703 
0 .8076 


Paraneter 
estinates 
-0 .846 
1 .029 
-0.118 
0.144 


Std  dev  .s  of 
estinates 

0 . 834 

1 389 
0 . 082 
0.118 


1 

i 


Stages  run  2100  Error  = 0.8271 

Std.  dev.  of  error*  0.6764 


Paraneter 
est i na  te s 
-0.805 
0 974 
-0  106 
0.131 


Std.  dev.s  of 
est i nates 

0 . 700 

1 . 235 
0 . 083 
0 . 090 


Stages  run  2400  Error  * 0 7931 

Std.  dev.  of  error*  0.7122 


Paraneter 
est i nates 
-1.031 
0.802 
-0.101 
0.151 


Std.  dev.s  of 
est i nates 
1141 
t 023 
0.155 
0 134 


Stages  run  2700  Error  * 
Std  dev . of  error* 


0.7719 
0 7994 
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Poroneter 

S td . dev  .s  of 

est  i piate s 

est i nates 

-0.701 

0 . 904 

0 429 

1 . 220 

-0  087 

0 . 124 

0 129 

0 . 104 

Stages  run 

3000 

Error  * 

Std  . 

dev. 

of  error* 

Parameter 

Std 

dev . f of 

est i nates 

est i nates 

-0  860 

0.810 

0 . 409 

0 . 736 

-0  .092 

0 131 

0.135 

0 . 120 

0 .3953 
0 .4421 
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System  order  n = 2 


The  sy  st  e n parameters  ar?(  in  the  ordera  1-  . . < an<bl < . . < bn)  : 
-1.425 
0.496 
0.000 
-0.102 
0.173 
0 0 00 

Nomina!  noise*1  Gaussian  Mean®  0.00 

Contaminating  noise:  Gaussian  Nean*  0.00 

Contamination  level  = 0.10 

The  Hampel  parameters  are:  si  = 1.00 

dl  = 0 . 1 5e  01 

ucutof  = 0 . 1 7 e 


St  d . dev .*  1.00 

1 

St  d . de  v . = 1 0 . 00 


s 2 = 0.00 
d 2 = 0 17e  38 
38 


Stages  run  300  Error  = 
Std.  dev.  of  error= 


1 .8582 
2 .2992 


Paraneter 
estimates 
-0 .396 
0.819 
-0.118 
0.096 


Std.  dev.s  of 
estimates 
2.529 
3 . 545 
0.171 
0 . 293 


Stages  run 

600 

Error  = 

1 . 3361 

Std. 

de  v . 

of  error® 

0 .9530 

Parameter 

Std 

dev.s  of 

est i mates 

estimates 

-0.406 

1.015 

0 .956 

1 965 

-0.142 

0 . 150 

0 032 

0 . 239 

Stages  run 

900 

Error  ® 

1 .2934 

Std. 

dev. 

of  error® 

0 .7963 

l 

* 


Paraneter 
e s t i mates 
-0.552 
0 .995 
-0  096 
0 077 


Std  dev.s  of 
est i nates 
0 985 
1.819 
0 . 134 
0 168 


Stages  run  1200  Error 


1 . 2339 
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Paraneter 
e s t i nates 
-0  948 
0 .348 
-0.105 
0.131 


Std  . dev  s of 
e st i nates 

0 . 670 

1 . 536 
0 . 075 
0 095 


Stages  run  3000  Error  3 0.5314 

Std.  dev.  of  error3  0.3059 


P ar  a ne ter 
est  i nates 
-1.111 
0.538 
-0.101 
0 137 


Std.  dev .s  of 
est  i nates 
0 . 608 
0 . 596 
0 . 162 
0.133 
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S ys t e«  order  n = 2 


The  system  paraneters  are(in  the  order  a 1 / . . < an < b 1 < . . < b n ) : 
-1.425 
0.496 
0 .000 
-0.102 
0.173 
0 .000 


Nominal  noise: 

Gaussian 

Mean* 

0 . 00 

St  d 

dev 

.*  1 

. 00 

Contaniniting  noise: 

Gaussian 

Mean* 

0 . 00 

St  d 

dev 

. *10 

. 00 

C on t an i n a t i on  level  = 

0.10 

The  Hanpel  paraneters 

are: 

si  = 1 . 

00 

s 2 

= 0 . 

00 

dl  =0 

lOe  01 

d 2 

= 0 . 

1 7 e 

38 

uc  u to  f » 0 

. 1 7e  38 

Stages  run 

300  Error  * 

2 .7769 

Std . 

dev . of  error* 

7 .6474 

Par  ane  ter 

Std.  dev . s of 

e st i nates 

est i nates 

1 .428 

10.316 

1 .633 

5 . 923 

-0.182 

0.281 

-0 .250 

1.474 

Stages  run 
Std  . 

600 

dev. 

Error  = 

of  error* 

1 .2755 
1 . 1086 

Paraneter 
estimates 
-0.502 
0 .926 
-0.115 
0 .070 

Std.  d ev . s of 
estimates 
1 . 245 
1 989 
0 . 142 
0 . 206 

Stages  run 
Std. 

900 
de  v . 

Error  * 

of  error* 

1.1136 

0.8071 

Paraneter 
est i nates 
-0.409 
0.798 
-0  092 
0.074 

Std.  d ev . s of 
est i na  te  s 
0 . 808 
1 . 600 
0 . 127 
0.147 

Stages  run 

1200 

Error  = 

1 0546 

222 
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Std.  dev.  of  error3  1.0074 

Parameter  Std.  dev.s  of 
estimates  estimates 

-0.71?  0.996 

0.982  1.774 

-0115  0.131 

0.102  0.171 


Stages  run  1500  Error  = 0.9873 

Std.  dev.  of  error*  1.0561 

Parameter  Std.  dev.s  of 
estimates  estimates 

-0  790  0.974 

0.885  1.815 

-0  090  0.132 

0.144  0.158 


Stages  run  1800  Error  3 0.7576 

Std.  dev.  of  error3  0.7085 


Parameter  Std.  dev.s  of 
estimates  estimates 

-0.944  0 597 

0.908  1.309 

-0.108  0.074 

0.138  0.121 


Stages  run  2100  Error  = 0.8180 

Std.  dev.  of  error3  1.0616 

Parameter 
est i mates 
-0.951 
0.995 
-0.103 
0 129 


Stages  run  2400  Error  3 0.6734 

Std.  dev.  of  error3  0.5808 

Parameter 
est i mates 
- 1 0 10 
0 7 44 
-0  090 
0 14? 


Stages  run  2700  Error  3 0.7686 

Std.  dev.  of  error3  0.7594 


Std.  dev.s  of 
estimates 
0.618 
1 092 
0 . 108 
0 105 


Std.  dev.s  of 
estimates 

0 . 536 

1 . 839 
0 . 073 
0.108 


J un 
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System  order  n = 2 


The  system  parameters  are(  in  the  ordera  1»  . . < an  , b 1 - . . < bn  ) : 
-1.425 
0.486 
0 .000 
-0  102 
0.173 
0.000 

Nominal  noise:  Gaussian 

Contam<nating  noise:  Gaussian 

Contamination  level  * 0 10 

The  Hampel  parameters  are: 


Stages  run  300  Error  = 1.2262 

Std.  dev.  of  error=  0.8646 

I 

Parameter  Std  dev.s  of 
estimates  estimates 

-0.013  1 . 189 

0.291  1 288 

-0.111  0.153 

1 0.048  0.255 


Stages  run  600  Error  * 1.2445 

Std.  dev.  of  error*  0.9071 

Std.  dev.s  of 
estimates 
1 167 

1 . 796 
0 . 128 
0 . 195 


900  Error  = 1.3012 

dev.  of  error*  1.2795 

Std.  dev.s  of 
estimates 
1 . 954 
1819 
0 111 
0 221 


i J 

Stages  run  1200  Error  * 1 0301 


Parameter 
e s t i mates 
-0.595 
0 8 78 
-0  ,'l  15 
0 049 

i 

Stages  run 
Std 

1 

Parameter 

| 

est i mates 
-0.726 
0.642 
-0.105 
0 041 


si  • 1.00  s 2 * 0 00 

dl  * 0 . 7 Oe  00  d2  * 0 17e  38 

uc  u to  f = 0 . 1 7e  38 


Neon*  0.00  Std.  dev*  1.00 

Mean*  0.00  Std.  dev. *10. 00 
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Std  daw  of  error*  0.9563 


Poraneter 
est  i nites 
-0  8 14 
0 663 
-0.124 
0 092 


Std  dev.s  of 
est  i nates 

0 985 

1 775 
0.127 
0.182 


Stages  run  1500  Error  * 0.9741 

Std  dev  of  error*  0 8148 


Paraneter 
est  i nates 
-0  728 
0 6 11 
-0.189 
0 050 


Std  dev.s  of 
est i na  te  s 

0 . 989 

1 352 
0.464 
0.439 


Stages  run  1800  Error  = 0.7746 

Std.  dew.  of  error*  0.7524 


Para  n e ter 
est i notes 
-0  879 
0 871 
-0.091 
0.134 


Std  dews  of 
est  i nates 

0 . 588 

1 . 375 
0.105 
0.115 


Stages  run  2100  Error  = 0.8052 

Std  dew.  of  error*  0.9588 


Para  r.e  ter 
est i nates 
-0  655 
0 967 
-0  069 
0 14  1 


Std  dev.s  of 
e s t i na  te  s 

0 . 637 

1 627 
0.141 
0.104 


Stages  run  2400  Error  * 0.7561 

Std  dew  of  error*  0.7479 


Paraneter 
estimates 
- 1 065 
0.636 
-0.105 
0.131 


Std  dev.s  of 
est i na  te  s 

0 . 697 

1 . 400 
0.106 
0.107 


Stages  run  2700  Error  = 0.6503 

Std.  dew  of  error*  0 5245 
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System  order  n * 


The  systen  parameters  are(in  the  ordera  1 / . / an  < b 1 < . . j bn  ) : 

-1.425 
0 . 4 96 
0 000 
-0.102 
0.173 
0 .000 

Nominal  noise:  Gaussian 

Contaminating  noise:  Gaussian 

C on t an i n a t i on  level  3 0.10 

The  Hampel  parameters  are: 


Mean=  0.00 
Mean3  0.00 


Std.  dev.=  1 
Std.  dev.310 


si  3 1.00 
dl  3 0 . 1 2e  01 

U C U t 0 ♦'  3 


s 2 =-0.27 
d 2 3 0 . 35e 
0 . 79e  01 


Stages  run 

300 

E r ro  r 3 

1 .5421 

Std  . 

dev. 

of  error3 

1 .5504 

Parameter 

Std 

dev.s  of 

est i na  tes 

estimates 

-0.419 

1.816 

0.430 

2.575 

-0.131 

0.184 

0.044 

0.291 

Stages  run 

600 

Error  = 

1.2189 

Std. 

dev. 

of  error3 

0 .7402 

Parameter  Std.  dev.s  of 


e s t i mates 
-0.242 
0.705 
-0  0 38 
0 051 


estimates 
1 . 069 
1 . 429 
0.121 
0 226 


V 

Stages  run  900  Error  3 
Std.  dev.  of  error3 


2.0823 

5.4919 


Paraneter 
est i nates 
- 1 .950 
0 5 12 
0 . 365 
0.542 


Std.  dev.s  of 
est i mates 
8.043 
1.645 
2 . 464 
2.416 


Stages  run  1200  Error 


1.0  178 

228 


. 00 

. 00 

0 1 
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Std 

dev 

of  error= 

Parameter 

Std 

dev .s  of 

e st  i 

mates 

est  i nates 

-0 

S 51 

1 12  1 

0 

9 19 

1 370 

-0 

1 06 

0.116 

0 

1 25 

0.185 

Stages  run 

1500 

Error  = 

Std. 

dev. 

of  error= 

Parameter 

Std 

dev  s of 

e st  i 

nates 

est i nates 

-0 

.9  15 

1 304 

0 

933 

1369 

-0 

1 07 

0 103 

0 

1 52 

0.231 

Stag 

es  run 

1800 

Error  = 

Std. 

dev. 

of  error= 

Parameter 

Std 

d ev . s of 

e 3t  i 

nates 

est i nates 

-0 

975 

0.619 

0 

7 99 

1.195 

-0 

111 

0 072 

0 

139 

0.125 

Stages  run 

2100 

E r ro  r = 

Std 

dev 

of  errors 

Parameter 

Std 

dev . s of 

e a t 1 

mates 

est i na  tes 

- i 

0 12 

0 . 628 

0 

855 

1 . 409 

-0 

1 03 

0.065 

0 

136 

0.104 

Stages  run 

2400 

Error  = 

Std. 

dev 

of  error* 

Parameter 

Std 

dev . s of 

e 3t  : 

mates 

e 

st i nates 

- 1 

074 

0 627 

0 

623 

0 972 

-0 

103 

0 . 077 

0 

1 42 

0.102 

Stages  run 

27  0 0 

Error  = 

Std 

dev. 

of  error® 

0.9892 


0 7 29  1 


0 7615 


0.6415 


2 804  9 


.V  • 


r 


■ 
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Parameter 

Std  dev  .s  of 

est  ifiQtes 

est  i no  te  s 

-0  540 

3 . 037 

-2.834 

18.138 

-0.372 

1 432 

-0  074 

1 . 154 

Stages  run 

3000 

Error  = 

Std  . 

dev. 

o f 

error* 

Para  he  ter 

Std 

dev .s  of 

e st  i no  te  s 

e s t i 

notes 

- 1 .118 

0 . 

567 

0 527 

0 

814 

-0 .085 

0 . 

088 

0.148 

0 . 

084 

0 5742 
0 .3782 


} 


^ - 
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System  o r d e r n = 


The  system  parameters  are < in  the  ordera  1/  . . - an,  b 1 - . . / bn  ) : 
- 1 425 

0 4*<J 

0 0 00 
-0  102 
0 1 73 
0 0 00 


G a us s i an 


Nor>  i na  t noise: 

Contaminating  noise:  Gaussian 

Contamination  level  = 010 

The  Han pel  parameters  are: 


Hean=  0.00 
Hean=  0.00 


si  = 100 
dl  = 0 . 25e  01 


St  d . dev  .=  1 00 

Std  dev.  = 10  00 


s 2 = -0  50 
d 2 = 0 45e  0 1 


ucutof  = 0 . 9 5 e 01 


S t a g e 3 run 

300 

E r r o r = 

1 8983 

Std. 

dev. 

of  error= 

2.4334 

Para  we  ter 

Std 

dev . 5 of 

e 3 1 i nates 

est i nates 

-0  9 48 

2.394 

0 3 14 

3 983 

-0  159 

0.159 

0 040 

0.392 

Stages  run 

600  Error  = 

1 .7452 

Std 

dev.  of  e r r o r = 

1 .9129 

Parameter 

Std  dev  .s  of 

est i nates 

est i nates 

0 5 11 

2 930 

0.731 

1 629 

-0  0 95 

0.150 

-0  .0  98 

0 630 

Stages  run 

900 

Error  = 

1 .7048 

Std 

dev. 

of  error= 

1 .6624 

Para  r*  e t e r 

Std 

dev.s  of 

est i nates 

e 

st i nates 

- 0 3 16 

2.011 

1 270 

2 675 

• | -0088 

0.159 

0 124 

1 1 

0 28  3 

* I 

I 

i Stages  run 

1200 

E r r o r = 

1 7496 

231 


, t. 


: 


r 
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Std 

dev.  of  error®  2.2380 

Puraneter 

Std  dev  .s  of 

est i nttes 

est i nates 

-0  £79 

3 . 102 

1 .307 

2.541 

-0.152 

0.430 

0.031 

0 664 

Stages  run 

1500 

E r r o r = 

1 0392 

Std  . 

dev. 

of  error® 

0 .6749 

Parameter 

Std 

dev .s  of 

est i hates 

est i nates 

-0.958 

1.186 

0 844 

1.327 

-0.139 

0.112 

0 147 

0 . 186 

Stages  run 

1800 

Error  = 

0 8679 

Std 

dev. 

of  error® 

0 .6904 

Paroiieter 

Std 

dev  ,s  of 

est i nates 

est i nates 

-0 .790 

0.820 

0 991 

1 . 226 

-0.124 

0 . 084 

0.127 

0.129 

Stages  run 

2100 

Error  = 

0 .9006 

Std  . 

dev. 

of  error® 

0.7942 

Paraneter 

Std 

dev .s  of 

est  iMites 

e 

st i nates 

-0  865 

0 816 

1 056 

1 . 425 

-0.122 

0.064 

0.115 

0.105 

Stages  run 

2400  Error  = 

0 .7024 

Std 

dev  of 

errors 

0 .4960 

Par  a n e t e r 

Std.  devs  of 

est i nates 

e s t i 

pm  te  s 

-0  8 2! 

0 . 

636 

0 775 

0 . 

917 

-0  0 97 

0 . 

108 

0.130 

0 . 

107 

Stages  run  2700 

Error  = 

0 .7800 

Std.  dev. 

of  error® 

0 644  3 
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Par  a weter 

S td 

d e v . s of 

est i hates 

est 

i nates 

-0  8 9? 

0 

. 83  8 

0 410 

1 

.169 

-0  112 

0 

. 080 

0 1 27 

0 

.106 

Stages  run 

1000  Error  = 

S t d . 

dev.  of  error= 

Par  aneter 

3td  dev .5  of 

est  i nates 

est i nates 

- 1 .0  50 

0 . 32? 

0 6 99 

1.430 

- 0 111 

0.061 

0 114 

0 106 

* 


\ 

i 


0 ?692 
0 819? 
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I . Introduct  ion 


With  the  continuing  evolution  of  digital  processing  ele- 
ments the  task  of  discretizing  signals  in  communication  and  con- 
trol systems  is  becoming  increasingly  important.  Perhaps  the 
most  widely  used  method  of  converting  discrete-time  analog  data 
to  digital  form  is  a simple  amplitude  quantizer.  Often  a uni- 
form quantizer  is  used  due  to  its  simplicity  and  general  good 
perf  ormance[l , 2]  . However,  if  the  class  of  input  signals  can  be 
restricted  by  a priori  information,  the  designer  should  be  able 
to  improve  performance  by  constructing  a quantizer  matched  to 
the  characteristics  of  the  input  signal.  Typically,  the  a 
priori  information  is  the  specification  of  the  cumulative  proba- 
bility distribution  of  the  input  signal.  However,  in  many  cases 
the  distribution  of  the  input  signal  is  only  approximately 
known,  so  that  the  design  task  is  complicated.  In  these  cases  a 
robust  quantizer  may  provide  the  most  acceptable  performance. 
The  robust  quantizer  will  guarantee  the  best  possible  perfor- 
mance commensurate  with  the  available  knowledge  of  the  input 
distribution. 

This  paper  considers  robust  quantization  when  the  a priori 
information  consists  of  knowledge  that  the  input  signal  distri- 
bution is  unimodal  and  satisfies  a generalized  moment  con- 
straint. The  restriction  to  unimodal  distributions  is  a natural 
one  and  results  in  least-favorable  distributions  much  more  real- 
istic than  those  found  in  similar  problems [3,4] . The  general- 
ized moment  condition  includes  the  possibility  of  using  standard 
moments,  such  as  variance,  as  well  as  approximations  to  other 
forms,  such  as  percentile  constraints.  With  these  constraints 
defining  the  admissible  distributions  and  with  performance  meas- 
ured by  a mean  weighted  quantization  error  criterion,  the  prob- 
lem is  to  find  the  least-favorable  distribution  and  the  best  N- 
level  quantizer  in  the  minimax  sense.  Section  II  of  the  paper 
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presents  the  formal  statement  of  the  problem.  In  section  III 
the  existence  of  a solution  is  demonstrated,  necessary  condi- 
tions are  presented  and  an  algorithm  for  finding  the  solution  is 
given.  Section  IV  considers  the  performance  of  the  robust  quan- 
tizer for  a specific  choice  of  the  error  weighting  function  and 
moment  constraint. 

II.  Problem  Stat ement  and  Preliminaries 

Let  X denote  the  real  random  variable  to  be  digitized  and 
let  F denote  any  of  the  possible  cumulative  distribution  func- 
tions of  X.  The  distribution,  F,  is  restricted  by  the  general- 
ized moment  constraint, 

G(F)  = /°p(s)dF(s)  i c,  (1  ) 

-co 

where  p is  a convex,  monotonically  increasing  function  of  Jsj. 
The  most  typical  choice  would  be  a moment,  p(s)  = js|P.  We  also 
impose  the  condition  that  the  distribution  be  unimodal  as  de- 
fined by  Feller[5] : 

Definition:  A distribution  function  F is  called  uni- 

modal with  mode  at  the  origin  if  and  only  if  the  graph 
of  F is  convex  in  [-ao,0)  and  concave  in  (0,co].  The 
origin  may  be  a point  of  discontinuity,  but  apart  from 
this,  unimodality  requires  that  there  exist  a density 
f which  is  monotone  in  [-<x>,0)  and  in  (0,<x>].  (intervals 
of  constancy  are  not  excluded.) 
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Denote  the  set  of  distributions  satisfying  these  two  constraints 
by  D,  which  we  take  to  be  a set  in  the  normalized  space  of  func- 
tions of  bounded  variation  on  the  extended  real  line 
(NBV  [-<e,  gd]  ) . Since  D contains  only  probability  distribution 
functions,  it  is  a subset  of  the  unit  ball  in  the  space,  which 

•Jt 

is  compact  in  the  weak  topology  by  the  Banach-Alaoglu 
theorem[6] . It  is  easily  verified  that  D the  admissible  set  of 
distributions  is  weak*  closed.  Let  D1  be  the  set  of  distribu- 
tions satisfying  (1),  and  be  the  set  of  unimodal  distribu- 
tions. 

Lemma  1:  D1  is  weak*  closed. 


Proof:  Since  p is  convex  and  bounded  from  below,  there  exists 

an  increasing  sequence  pi  of  bounded  continuous  functions  con- 
verging pointwise  to  p.  It  follows  from  the  definition  of  the 
weak*  topology  that  the  sets  d|  consisting  of  the  distributions 
satisfying  ( 1 ) with  p ^ instead  of  p are  weak*  closed.  Also,  by 
the  monotone  convergence  theorem  the  intersection  of  the  must 
be  the  same  as  D.  Thus,  by  the  infinite  intersection  property,  D 
is  weak*  closed. 

Lemma  2:  is  weak*  closed. 

Proof:  Suppose  Fi  is  a sequence  of  unimodal  distributions  weak 
convergent  to  a distribution  F.  We  must  show  that  F is  unimo- 
dal. Since  the  F are  continuous,  except  possibly  at  the  ori- 
gin,  weak  convergence  is  equivalent  to  pointwise  convergence. 
Each  F^  is  convex  for  negative  argument  and  concave  for  positive 
argument;  accordingly,  the  pointwise  limit  of  the  functions  must 
similarly  be  convex  and  concave  — thus  unimodal. 

The  intersection  of  two  closed  sets  is  closed,  so  that  D is 
weak*  closed.  Thus,  being  a subset  of  the  unit  ball,  D is  also 


* 

wea>c  compact.  Furthermore,  since  G is  linear  in  F and  since 
convex  combinations  of  unimodal  distributions  are  unimodal  dis- 
tributions, D is  also  a convex  set. 

Remark:  In  order  to  guarantee  existence  of  solutions 
to  an  optimization  problem  the  admissible  set  must 
usually  be  compact.  Since  compactness  with  respect  to 
the  norm  topology  is  a very  severe  requirement  in  a 
function  space,  most  sets  of  interest  will  not  be  norm 
compact.  Since  Weak  compactness  is  much  less 
stringent,  it  is  a more  useful  starting  point  for 
proving  existence  of  solutions. 

A N-level  quantizer  q is  a device  with  input  X and  output  Y 
defined  by 


q(x) 


1 y1  xe  [-00,  bi ) 

j Y±  xe  fb±_1  , b^) 

t 

j yN  xe[bitoo]  , 


(2) 


with  bQ  £ y1  £ y2  ^ ...  £ yN  £ bR  (see  below  for  the  definition 
of  bQ  and  bj^).  The  bi  have  been  called  transition  values  and 
the  yi  representation  values.  The  admissible  values  of  the  2N-1 
quantizer  parameters  clearly  define  a convex,  compact  set  in 
r2N— 1 a call  this  set  Q. 

A number  is  associated  with  each  quantizer,  q,  and  each 
distribution,  F,  which  measures  the  fidelity  of  the  quantizer's 
representation  of  the  input  signal.  This  number  is  defined  by 
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(3) 


i 


E(q,F)  = /V(  s)dF(s) 

-SO 


where  the  integrand  g*(s)  is  an  upper  semlcontinuous  function 
defined  by 


g*(s) 


g(bo-yi ) 


se  [-00,  b0] 


?( s-yf) 

g(bt,-yN) 


f rr  ( V> \r  . ^ 


se(bi_i»bi)  i=1,...,N 

se  [hjj.co] 

s =b ^ i= 1 , . • • t N— 1 , 


with  g(t)  a convex  weighting  function,  monotonic  in  ItJ.  The 
numbers  bQ  and  define  the  input  dynamic  range  of  interest. 
The  assumption  being,  if  the  signal  exceeds  these  values  no  ad- 
ded penalty  should  be  assessed  for  the  increased  representation 
error.  The  function  E(q,F),  considered  as  a functional  on  D, 
can  be  shown  to  be  weak  upper  semi-continuous.  Considered  as  a 
functional  on  Q it  is  continuous.  However,  while  E (q,F ) is  con- 
cave (linear)  in  F,  no  similar  statement  can  be  made  with 
respect  to  q. 


Remark:  The  saturation  of  the  weighting  function  is 
included  for  both  aesthetic  and  mathematical  reasons. 

If  the  rate  of  increase  of  g is  greater  than  that  of 
p,  the  problem  does  not  have  a solution.  If  they  are 
the  same,  then  the  cost  is  equal  to  c,  the  value  of 
the  constraint. 

Armed  with  the  above  definitions  the  statement  of  the  prob- 
lem is  simple: 
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inf  sup  E ( q , F ) 
qeQ  FeD 


(5) 


III.  Main  Re suit s 


Before  attempting  to  find  explicit  solutions  to  the  prob- 
lem, we  verify  the  existence  of  least-favorable  distributions 
and  robust  quantizers,  so  that  our  search  shall  not  be  in  vain. 

Theorem:  For  an  arbitrary  quantizer  q in  Q there  exists  a max- 
imizing distribution  function  in  D.  Also,  there  exists  a minim- 
izing quantizer  for  the  resulting  minimization  problem. 

Proof:  An  upper  semicontinuous  function  achieves  its  supremum 
on  a compact  set[6] . Since  E( q,F)  is  weak*  upper  semicontinuous 
and  D is  weak*  compact,  the  conditions  are  satisfied  for  any 
quantizer.  The  second  part  of  the  proposition  follows  since  Q 
is  compact  and  the  sup  E(q,F)  is  lower  semicontinuous  on  Q. 

As  usual,  finding  solutions  to  a problem  is  a bit  more  dif- 
ficult than  proving  their  existence.  In  this  problem,  knowing 
that  there  exists  a distribution  solving  the  problem  is  of  some 
help  in  itself  for,  as  Max[7]  showed,  when  finding  the  optimum 
quantizer  for  a given  distribution  function,  the  optimal  quan- 
tizer parameters  satisfy 

bi  = (yi  + yi+l)/2  i=1 ,2,...,N-1  (6) 

no  matter  what  the  distribution.  This  same  set  of  equations 
will  hold  for  our  problem;  thus,  reducing  the  number  of  unknown 
quantizer  parameters  by  almost  one-half.  The  primary  tool  which 
will  be  used  to  provide  necessary  conditions  that  the  remaining 
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quantizer  parameters  must  satisfy  is  the  Lagrange  Duality 
theorem[7] . This  theorem  is  stated  below  in  a form  appropriate 
to  the  current  problem. 

Theorem  2 (Lagrange  Duality) : Let  C be  the  convex  cone  defined 
by  D.  Suppose  that  (<30,Fq)  is  a solution  to  the  robust  quanti- 
zation problem  and  u0=E(qo»Fo)  is  the  rninimax  value  of  the 
fidelity  criterion.  Then  the  following  equation  holds: 

co 

un  = min  min  max  [ E(q,F)-X1J'  dF(  s)-X_g  (f  )+X.  +X9c  ]»  (7) 
qeQ  X^O  FeC  1 -co  Z 1 * 

where  the  multiplier,  X=(X1>x2).  Call  the  minimizing  value  of 
X,  X®.  Either  X®=0  or  the  corresponding  constraint  is  satisfied 
with  equality,  that  is, 

xO(g(f0)-c)  = xO(;®dF0(s)_i)  = o.  (a) 


Note  that  the  problem  as  stated  is  symmetric  about  the  or- 
gin  and  there  will  be  no  loss  in  generality  by  considering  only 
the  non-negative  half-line  as  the  domain  of  definition  for  the 
problem.  We  need  only  set  bj^  or  y(N+l)/2  equal  to  zero 
depending  on  whether  N is  even  or  odd,  respectively.  Also, 
renumber  the  parameters  so  that  the  b or  y set  to  zero  is  now 
subscripted  by  zero  and  the  new  value  for  N is  the  old  one  di- 
vided by  two  or  one  plus  the  old  value  divided  by  two  as  ap- 
propriate. Having  done  this,  consider  the  maximization  with 
respect  to  F in  (7).  We  can  write  the  terms  which  are  a func- 
, ) tion  of  F as, 
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max  £ Cg*( s)-x1-x2p( s)]dF( s) . 


(9) 


Since  F is  restricted  to  be  concave,  it  can  increase  at  most 
linearly.  Thus,  a necessary  and  sufficient  condition  for  (9)  to 
be  finite  is  that 


H ( x)  [g*(s)-Xrx2p(s)]ds^O,  V x^O.  (10) 

If  this  condition  holds,  then  the  maximum  value  of  (9)  is  zero. 
Using  (10)  an  equivalent  statement  of  the  problem  is 

min  min  (X-,  + X9c),  (H  ) 

X^O  qeQ 

subject  to  the  condition  given  by  ( 1 0 ) . We  must  be  able  to 
determine  if,  for  a given  X,  there  exists  a set  of  quantizer 
parameters  making  (10)  hold.  If  we  can  do  this,  the  problem  is 
reduced  to  a two  dimensional  optimization  problem,  and  there  are 
any  number  of  algorithms  that  can  be  used  to  solve  it. 

Inequality  ( 1 0 ) will  be  satisfied  if  and  only  if  all  of  the 
local  maxima  of  H(x)  are  less  than  or  equal  to  zero.  A neces- 
sary condition  of  the  local  extrema  of  H is 

g*(x)  - X1  _ X2p(x)  = 0.  (12) 

Because  of  the  form  of  g*,  this  equation  can  have  at  most  2N+1 
roots,  N of  which  are  local  minima  or  inflection  points  and  N+1 
which  are  candidates  for  local  maxima  (They  may  also  be  inflec- 
tion points.).  The  equations  for  these  N+1  points,  x^^  are: 
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and 


gtYi-Xj,)  - X1  -X2p(xi)  = 0 i=1,2,...,N  (13) 


g(yN— bjj)  - x1  - x2p(xn+1  ) = o b^£xN+1.  (14) 


Remark:  If  the  number  of  quantization  levels  in  the 

original  problem  was  odd,  then  y1=o  and  (13)  holds  for 
i=  2, 3, . • . ,N. 

We  need  to  determine  if  there  exist  quantizers  such  that  all  of 
the  local  maxima  of  H are  less  than  or  equal  to  zero.  Consider 
the  construction,  described  below,  for  generating  a set  of  quan- 
tizer parameters: 

1 ) Starting  with  y1  and  continuing  iteratively  through 
the  i's,  pick  the  values  of  the  Y^'s  so  that  the  value 
of  H(x)  is  exactly  zero  at  the  points  xi(  i=1,2,..*,N. 

2)  If  H(xn+jJ)  ^ 0,  the  constructed  quantizer  satisfies 
(10).  Alternatively,  if  H(XN+1 ) > 0,  then  the  con- 
struction procedure  fails. 

This  procedure  will  always  find  a quantizer  satisfying  (10)  if 
one  exists.  We  give  the  following  lemma  to  make  this  point  pre- 
cise. 

Lemma  3:  For  a given  value  of  X , if  the  set  of  quantizers 

satisfying  inequality  (10)  is  not  void,  the  construction  tech- 
nique described  above  will  generate  a quantizer  in  the  set. 


The  proof  of  this  lemma  is  not  difficult,  but  is  quite  messy. 
Therefore,  we  will  provide  only  an  outline  of  the  proof  First, 
observe  that  no  matter  what  values  and  take,  we  can  always 
find  values  for  the  y^  making  part  one  of  the  construction  true. 
Thus,  we  only  need  prove  that  no  other  choice  of  the  y^  will 
make  the  second  part  true  if  our  choice  fails.  The  approach 
taken  is  to  compute  the  variation  of  the  N+1  maximum  with 
respect  to  changes  in  the  noting  that  the  y's  can  only  be 
moved  to  the  left  and  that  moving  any  one  of  them  left  implies 
some  motion  to  the  left  of  all  of  the  others  of  higher  index. 
The  variation  shows  that  any  such  motion  will  cause  the  value  of 
the  N+1  maximum  to  increase,  thus  continuing  to  be  greater  than 
zero. 


By  using  the  construction  procedure,  the  computer  program 
.necessary  to  compute  the  quantizer  parameters  and  the  minimax 
value  of  the  error  is  quite  short  and  requires  a minimal  amount 
of  central  processor  time. 


IV.  Example 


To  illustrate  the  behavior  of  solutions  to  the  problem,  we 
consider  a simple  example.  Let  the  functions  g and  p both  be 
quadratic  and  let  the  total  number  of  quantization  intervals  be 
four  with  -bg_b4=l . The  values  of  the  quantizer  parameters  and 
the  worst  case  error  were  determined  for  several  values  of  the 
moment  constraint.  These  are  tabulated  in  table  1 for  various 
values  of  ratio  between  the  saturation  amplitude,  b^  and  the 
standard  deviation  of  the  signal  constraint.  Also  given  are 
values  for  the  worst  case  error  for  the  cases  from  [l  ] and  [2] 
in  which  the  distribution  was  not  required  to  be  unimodal. 
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Uniraodal 


Not  Uniinodal 


s.d. 


1.0 


2/3 


0.5 


0.17 


0.17 


0.16 


0.49 


0.48 


0.47 


V2 


0.81 


0.80 


0.77 


error 


Vi 


.0303 


.0300 


0.25 


0.25 


0.50 


0.50 


.0293 


0.20 


0.43 


V2 


0.75 


0.75 


0.67 


error 


.0625 


.0625 


.0589 


1/3 


0.1  3 


0.33 


0.62 


.0265 


0.12 


0.  32 


0.50 


.040  3 


0.2 


.086 


0.25 


0.41 


.0165 


.070 


0.23 


0.38 


.0212 


0.1 


.040 


0.1  3 


0.22 


.0066 


.031 


0.14 


0.25 


.0066 


Table  _1 . Robust  Quantiser  Performance 
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APPENDIX  B 


ROBUST  PROPERTIES  OF  SOLUTIONS  TO  LINEAR-QUADRATIC  ESTIMATION 
AND  CONTROL  PROBLEMS 


In  this  note  we  hope  to  provide  insight  into  the  optimality 
properties  of  the  standard  solutions  to  a large  class  of 
linear— quadratic  estimation  and  control  problems,  when  they  are 
viewed  in  a more  general  context.  Of  importance  to  the  systems 
engineer  when  he  designs  am  estimation  and  control  system  is  the 
sensitivity  of  that  design  to  deviations  from  the  assumed  model 
of  the  environment.  Sensitivity  analysis  of  plant  parameters  is 
an  extensively  studied  discipline  for  linear  systems;  but, 
beyond  parametic  sensitivity  studies  of  the  effect  of  change:  in 
the  first  two  moments  of  the  noise  densities,  little  work  has 
been  devoted  to  the  sensitivity  of  such  systems  to  deviations 
from  the  assumed  statistical  model.  However,  in  the  statistical 
literature  this  problem  has  received  a great  deal  of  attention 
in  the  last  decade[l , 2, 3] , leading  to  a subarea  of  statistics 
which  we  refer  to  here  as  robust  design.  Loosely  speaking,  a 
robust  policy  may  be  described  as  one  which  performs  well  even 
though  the  actual  state  of  nature  deviates  "mildly"  from  the  en- 
vironment nominally  assumed  in  the  design.  We  put  the  word 
mildly  in  quotation  marks  since  it  is  a key  word  in  the  entire 
concept.  Quantifying  mildly  is  impossible  without  a good  physi- 
cal feel  for  the  noise  generating  process  and  its  inherent  con- 
straints. If,  for  example,  the  noise  is  generated  directly  from 
* well  understood  and  accurately  described  process,  such  as 
thermal  noise  in  a resistor,  then  tight  bounds  may  be  placed  on 
the  statistical  description  of  the  noise  process,  and  a mild  de- 
viation is  a very  small  change  in  the  process  statistics.  If, 
on  the  other  hand,  the  noise  to  be  modeled  is  a process  derived 
from  a source  which  is  not  very  well  understood  or  quantified, 
then  the  nominal  statistical  description  may  be  imprecise. 
Thus,  mild  deviations  from  the  nominal  description  may  represent 
large  changes  in  the  process  statistics. 

The  hope  of  the  engineer  who  adheres  to  the  philosophy  of 
robust  design  is  that  he  may  be  able  to  find  a policy  performing 
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within  a few  percent  of  optimality  for  the  most  likely  environ- 
ment while  protecting  against  performance  deterioration  of  much 
greater  magnitude  caused  by  ”mild"  deviations  from  the  assumed 
environment.  The  most  frequent  formalization  of  the  robust 
design  problem  is  as  a mathematical  game  between  the  engineer 
and  nature.  How  well  the  robust  design  can  be  effected  will 
depend  on  the  situation  at  hand  and  the  engineer's  ability  to 
pose  the  most  appropriate  game  to  be  solved.  A solution  to  the 
resulting  minimax  optimization  problem  consists  of  a most  robust 
policy  and  a least  favorable  state  for  nature.  If  the  least 
favorable  state  happens  to  correspond  to  the  most  likely  or  nom- 
inal state,  the  solution  is  particularly  meaningful,  since  no 
penalty  need  be  paid  for  robustness.  As  we  shall  show,  this  is 
the  situation  in  many  linear-quadratic  estimation  and  control 
problems.  First,  however,  we  need  some  basic  concepts  from  game 
theory. 

An  abstract  game  may  be  defined  as  a triple  (A,B,V),  where 
A and  B are  the  sets  of  possible  strategies  for  the  engineer  and 
for  nature,  respectively,  and  V is  a function  from  AxB  to  the 
real  line  which  measures  the  performance  of  a strategy  pair 
(a,b).  Nature  chooses  b to  maximize  V,  and  the  engineer  chooses 
a to  minimize  it.  A pair  (a0,b0)  is  called  a saddlepoint  pair 
for  the  game  if  V(aQfbg),  the  va^ue  the  9ame  satisfies 

V(*0.b)  ^ V(a0’b0>  V b € B (1) 

and 

V(a0,bQ)  i V(a,bQ)  V a e A.  (2) 

It  is  not  always  possible  to  find  a pair  satisfying  (1)  and  (2), 
and  in  such  cases  more  extensive  considerations  are  necessary. 
For  the  problems  considered  here  a saddlepoint  will  exist;  thus, 
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there  is  no  need  to  discuss  the  possible  additional  complica- 
tions (the  interested  reader  should  see  Ferguson [4] ) . 

This  setup  can  be  applied  directly  to  many  linear-quadratic 
estimation  and  control  problems.  Let  A contain  all  functions, 
both  linear  and  nonlinear,  of  the  observations  and  let  B contain 
all  probability  distributions  satisfying  the  standard  first  and 
second  moment  constraints.  Futhermore,  let  a^  correspond  to  the 
best  linear  solution  to  the  problem  and  let  bQ  correspond  to  the 
Gaussian  distribution  function  satisfying  the  moment  constraints 
(with  greatest  covariance  matrix,  if  the  second  moment  con- 
straint is  an  inequality).  Then,  inequality  (l)  requires  that 
the  linear  solution  perform  no  more  poorly  with  respect  to  any 
other  distribution  satisfying  the  constraints  than  it  does  with 
respect  to  the  Gaussian.  Since  the  problem  is  linear  and  the 
performance  measure  quadratic,  a linear  solution  yields  a value 
for  V that  is  only  a function  of  at  most  the  first  two  moments 
of  the  noise  distribution.  These  are  constrained  by  hypothesis, 
so  that  (l)  holds  with  equality.  Inequality  (2)  is  a statement 
that  the  linear  solution  must  be  optimal  with  respect  to  Gaus- 
sian distributions  for  the  noise.  Thus,  if  any  model  admits  the 
same  solution  to  both  the  linear-least-squares  problem  and  the 
optimal  Gaussian  problem,  then  this  same  solution  also  solves 
the  minimax- robust  problem.  A large  class  of  linear-quadratic 
estimation  and  control  problems  have  solutions  that  satisfy  this 
condition.  The  ease  with  which  this  fact  is  obtained  should  not 
belittle  its  importance.  The  philosophical  ramifications  of 
this  added  property  are  profound  for,  with  it,  we  can  provide 
added  justification  for  using  linear  policies  even  when  the 
noise  is  suspected  to  be  non-Gaussian.  Also  illuminated  is  the 
extreme  importance  of  the  constraint  on  the  second  moment  of  the 
noise.  It  is  this  constraint  on  which  the  efficacy  of  the 
linear  solution  rests  and  not  on  the  Gaussian  assumption.  As  we 
have  seen,  any  arbitrary  deviation  from  Gaussian  nois<«,  which 
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continues  to  satisfy  the  second  moment  constraint,  cannot  in- 


crease the  cost.  However,  there  exist  distributions  looking, 
for  all  the  world,  like  the  Gaussian  which  can  drive  the  cost 
arbitrarily  high;  for  example,  consider  a mixture  distribution 
with  the  Gaussian  being  the  true  distribution  99.9%  of  the  time 
and  a Cauchy  the  true  distribution  0.1%  of  the  time.  To  detect 
the  fact  that  the  variance  of  the  distribution  is  very  large  ( in 
fact  infinite)  would  require  many  thousands  of  observations; 
however,  any  given  sample  function  of  the  noise  could  drive  the 
estimation  error  or  control  cost  very  high. 


Although  the  above  results  have  escaped  the  general 
knowledge  of  the  engineering  community,  u.iey  are  not  entirely 
without  precedent.  In  fact,  for  the  Kalman  filtering  problem 
the  robust  property  of  the  solution  is  almost  as  old  as  the 
filter  itself,  having  been  pointed  out  by  Carlton [5].  In  various 
specific  problems [6, 7]  other  workers  have  used  similar  ideas  in 
a limited  context.  We  believe  appreciation  of  the  simple  result 
reported  in  this  note  and  an  application  of  the  ideas  in  a 
broader  research  context  can  be  of  major  importance  as  a direc- 
tion for  research  in  the  next  few  years. 
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